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SECTION  8 


GENERALIZATIONS  ON  DERIVATIVES  OF  BASE  FITTING  FUNCTIONS  AND  TRANSITION 
MATRICES 


The  first  derivative  of  the  Monomial  Base  is  given  by  Eq.  (22)  or  Eq.  (26)  as 


< t t V 


where 


i 0 1 0 0 

,0  0 2 0 


0 0 0 3 0 


For  an  arbitrary  base  we  have 


<f(t)  = ^ f(t)  V, 


Where  most  cases  considered  in  this  report  the  velocity  matrix  V will  be  a 


constant.  When  it  is  not  a constant,  higher  derivatives  will  yield 


<.f(t)  = < f(t)Vj  + < f(t)Vj 


or  using  Eq.  (4)  in  Eq.  (5) 


< f(t)  = <'f(t)Jv^  + V^*] 


with  similar  expressions  for  higher  terms.  When  V,  is  a constant  the  higher 
*--• t 


derivative  becomes 


< f(t) 


f(t)  V, 


<f(t) 


J 


J 


Lies. 


These  higher  derivatives  occur  in  state-variable  approximations,  for  example 
x(t)  = .<^f(t)  a (8) 


and  the  higher  derivatives  are 


x(t) 

x(t) 


t(t) 


l(t) 


> 


(9) 


Eq.  (4)  has  the  solution 


^f(t+to)  = ^f(to) 
where  the  transition  matrix 


V^(t-to) 


(10) 


Vf(t-to) 

4>  (t+to,  to)  = 0 = I + V (t-to)  + V^Ct-to)'^ 

^ ^ 2! 


(11) 


or  for  small  transitions  t-to  = At 


,2.  2 


4>^j(t+At,  t)  = I + V^At  + + 


(12) 


(13) 


and  for  linear-terms  only 

<t>jj(t+At,  t)  = I + V^t 
Returning  to  the  monomial  base  of  Eq.  (1)  and  Eq.  (3)  case  we  have  designating 

<(t  = <m(t)  (15) 

.(^m  = <m(t)  V (16) 

and 

* Vt 

< m(t)  = m(o)  0 =4^m(o)  <l>^(t,o) 

where 


(17) 


-1, 


<J>j.(t,o)  = ♦„(t,o)  = B(t)  = D "(t)  B(l)  D(t) 


(18) 
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I 


and  the  t-binomial  matrix  B(t)  is  given  by  Eq.  (158)  appendix  (A)  as 


2 3 

1 t t t . 


0 1 2t  St' 


= B(t)  =001  3t 


we  also  have 


<j)  (t)  = B(t)  =e'^’^ 
m 

i (t)  = B(t)  = = V*  (t) 

m m 

0 (t)  = B(t)  = e = v\(t) 

m m 


= B^'^'^^Ct)  = (t) 

m ^ m 

where  the  nilpotent  condition  holds 
= 0 

"*  dxd 

Consider  the  Poisson  polynomials  defined  by  Eq.  (134) 
<p(t)  = ^t  i ^ = </m(t)  A ^ 

Taking  the  first  derivative 
<p(t)  = <tV]#"^ 
or  by  Eq.  (22)  in  Eq.  (23) 

<p(t)  = .4p(t)  s V 


= <.p(t)v 
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hence  the  velocity  matrices  are  similar 

Vp  = (26) 

When  the  diagonal  factorial  matrix  of  Eq.  (25)  Section  (B)  is  used  in  Eq.  (26)  are 
obtained 


0 10  0 0 


0 0 
0 0 


0 0 


1 0 


1 0 


0 0 0 0 1 


0 0 


0 0 


0 1 
0 0 


Which  is  the  shift-matrix  of  Eq.  (32)  Section  (4), 


Eq.  (27)  is  the  companion  matrix  for  the  dynamical  system  described  by  the 
d-th  derivative  constant. 

x^*^^(t)  = constant 

The  transition  matrix  for  the  Poisson  base  is 

V t av4"^t 

*p(t)  =e  p =c£  = le 

or  by  Eq.  (19) 

<)>p(t)  = i B(t) 

Utilizing  the  diagonal  matrix  of  integer  factorials  and  its  inverse  in  Eq. 
(30)  one  obtains 

f 1 t t^  t^.  1 


^p(t) 


2 >3! 

^2. 


0 1 t t 


0 0 1 t 


0 0 0 1 


which  is  the  Taylor-Series  state  transition  matrix  of  Eq.  (177)  Section  (4). 
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For  linear  terms  in  At  the  transition  matrix  from  t to  t + At  is  by  Eq.  (13) 
or  Eq.  (31) . 

1 At  0 0 

<J>  (t+A,  t)  S 0 1 At  0 C 

P 0 0 1 At 

0 0 0 1 

and  for  At  = 1 

r 

1 1 0 0 0 (' 

0 110  0 

<t>  (t+1,  t)  S 0 0 1 1 0 

P 0 0 0 1 1 

0 0 0 0 1 


Equation  (33)  has  as  inverse 


110  0 0 
0 110  0 
0 0 110 
0 0 0 1 1 
0 0 0 0 1 


1-11-11 
0 1-1  1-1 

0 01-11 

0 0 0 1 -1 

0 0 0 0 1 


The  expression  of  Eq.  (29)  is  easy  to  show,  for  by  Eq.  (17) 
<m(t)  = < m(o) 
and  by  Eq.  (22) 

< p(t)  i = " p(o) 


/ P(t)  = <p(0)  F 
and  by  Eq.  (25) 


hence 


V = 4 V 
P 


<J)p(t)  = i i> 


Consider  the  dyadic  product  of  the  monomial  base  under  differentiation 


[m(t)^4^  m(t)]  = m^<(^m  + m'>^^ 


By  Eq.  (16) 

^ m = ^ mV 
and  transposing 
m > = 

On  the  (0,1)  interval  under  inner-product  Eq.  (39)  and  (40)  yield 


ID  =f^  m N<  mdtV  = H. 

^ r\  ' > 1 


lee 


and 


or 


J .1 


mXmdt  = V /»<'mdt  = V^H. 
o''  0 1 


lee 


(42) 


(43) 


(44) 


(45) 


1 ^ T 

><  'n)dt  = H.  V + V^H. 
0 dt'  lu  1 


lee 


Consider  the  states  in  the  monomial  base  given  by  Eq.  (110)  Section  (4) 
x(t)('d^  = T^(t)a(^ 


(46) 


(47) 


where  by  Eq.  (151)  and  Eq.  (75)  Section  (4) 


vt 

•c  tv. 


T (t)  = 

V 


-<tv‘ 


d-1 


(48) 


s tv 


and  for  5x5  case  we  have 


2 3 

1 t t'^  t"^ 


T (t)  = 

V 


4^ 


0 1 2t  3t 

002  2*3t  2-3-4t| 

0 0 0 0 2-3«4 

The  matrix  T^(t)  can  be  expressed  in  terms  of  known  factors  as 
T^(t)  = iT  B(t) 
and  hence  the  inverse  is: 

T‘\t)  = B(-t) 


(49) 


(50) 


(51) 
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(52) 


Taking  the  derivative  of  Eq.  (47) 

x(t)^  = T^(t) 

and 

T^(t)  = Ty(t)V 

and  Eq.  (53)  in  Eq.  (52)  yields. 
x(t)^  =T^(t)Va^ 

Solving  Eq.  (47)  for  the  constant  vector  a*^ 
a>  = T^^t)  x(t)N 
and  using  Eq.  (55)  in  Eq.  (54) 

^^(t)^  = T^(t)  V T‘\t)  x(t)  \ 
or 

x(t)  ^ x(t)^ 

where 

V = T (t)V  T"^(t) 

Using  Eq.  (50)  and  Eq.  (51)  in  Eq.  (58) 

= A B(t)  V B"^(t) 

Consider  next  the  inner-similarity  transformation  of  Eq.  (59),  namely 
B(t)VB"^(t)  = V(t) 


Eq.  (60)  can  be  shown  for  the  3x3  case  as 


- 

- 

• 

— 

1 

t 

t 

2 

0 

1 

0 

1 

-t 

0 

1 

2t 

0 

0 

2 

0 

1 

-2t 

0 

0 

1 

0 

0 

0 

0 

0 

1 

•n 

r 

= 

0 

1 

0 

0 

0 

2 

0 

0 

0 

Solving  Eq.  (57)  for  the  states 
V t 

x(t)')»  = ^ ^ y 


(53) 

(54) 

(55) 

(56) 

(57) 

(58) 

(59) 

(60) 

(61) 


(62) 
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or  using  Eq.  (59)  and  (60)  in  Eq.  (62) 

r V - 

x(t)^  =0  x(o)^ 


x(t)];>  = «i>^(t,o)  x(o)> 


By  the  analogy  of  Eq.  (29) 


t V i-'^t  Vt 

*jj(t,o)  = A B. 

♦j^(t,o)  = <P^  = t B(t) 


Vt  , 

e 


Note  that  the  V velocity  matrix  of  Eq.  (63)  is  the  same  as  the  Poisson 
velocity  matrix  of  Eq.  (27)  and  is 


= Av  = 


0 1 0 
0 0 10 
0 0 0 1 0 


0 1 
0 


Eq.  (67)  is  obvious  for  if  one  sets 

(d)  (d) 

x(t)  = x(o) 


one  obtains 

the 

state 

equation 

x(t) 

'\ 

1 

i 0 

1 0 

x(t) 

I 

1 

1 

0 1 0 

x^‘^\t) 

(- 

1 

! 

! 0 
' 0 

0 

0 

4^  — 

Velocities  for  Gram  Orthogonal  ! 

x^‘^^(t) 


The  orthogonal  polynomials  on  the  interval  (0,1)  are  given  by  Eq.  (336)  Section 
(1)  for  the  3x3  case  as 


<g  =< 


1 -1/2  1/6 

0 1 -1  = 4tBg 

0 0 0 


Taking  the  time  derivative 

< 8 = < tB^  = < tVB^  = </  gB’^VBg 
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The  general  terms  for  B as  functions  of  the  binomial  coefficients  etc. 
be  obtained  for  Section^(l)  to  use  in  Eq.  (71)  or  Eq.  (72). 


can 


The  orthogonal  polynomials  on  the  (0,1)  interval  are  given  by  Eq.  (337)  Section 
(1)  as 


s = < t 


-V3 

2V3 

0 


V5 

-6V5 

6-^5 


> 


= / tB 

N S 


and 


< s = < S 

where 


V = b“^  V B 
s s s 


V = 
s 


or 


V = 
s 


^ is 

given  by  Eq.  (360) 

Section  (1) 

hence 

1/2 

1/3 

r 

0 1 

0 

1 

-V3 

>/5 

V3/6 

V3/6 

0 0 

2 

0 

2^2 

-6V5 

0 

V5/30 

0 0 

0 

0 

0 

6V5 

0 

2V3 

0 

i 

i 

0 

0 2y|3^|5 

0 

0 

0 

j 

M 

-/♦-pm' 

One  also  has 

/s  = / gM"^  = <tBM 

^ ^ gg  ^ e 

hence 

< s = <sV 
where 


g gg 


V = M ^ B"^VB  M 
s gg  g g 11 


The  solution  is 


s(t)  = s(o)  ® 


(73) 


(74) 

(75) 


(76) 


(77) 


(78) 


(79) 


(80) 


(8i) 


or 


4 s(t)  = <s(o)  B'^p, 


(82) 


or 


^ s(t)  =</s(o)  B(t)  B 


(83) 


where  the  t-binomial  matrix  is  given  by  Eq.  (19).  Similar  expressions  are 
obvious  for  g (t). 


Time  Derivatives  of  the  Legendre  Polynomials  (Modified).  The  classical  Legendre 
polynomials  or  the  interval  (-1,1)  are  given  by  Eq.  (209)  Section  (2)  and  for 
the  unit  upper  triangular  case  called  the  Modified  Legendre  polynomials  of  Eq. 
(205)  Section  (2)  we  have  for  the  3x3  case. 


8 = < 


1 

0 

0 


0 

1 

0 


-1/3  j 
0 ! 
1 


= < tB. 


8 


(84) 


The  ensuing  analytical  terms  are  exactly  the  same  as  for  the  Gram  case  except 
the  matrix  entrees  are  different.  We  obtain  for  example 


= B"^V  B 


(85) 


V = 
8 


or 


V = 
8 “ 


(84) 

and  Eq. 

(22) 

Section 

(4) 

1 

- 

V. 

r 1 

0 

1/3 

0 

1 

0 

1 

0 

0 

1 

0 

0 

0 

2 

0 

1 

L ° 

0 

1 

0 

0 

0 

0 

0 

/-I 

• 

0 

1 

0 

0 

0 

2 

V 

0 

0 

0 

m 

(86) 


(87) 


a surprising  result  which  may  not  be  valid  for  the  d x d case,  since 

8 8 


<■  8(t)  = < g(o)B"^(^i  '^'^B 


(88) 


or  by  Eq.  (85) 

<g(t)  =.<:  8(o)e.''’'  = <8(o)B(t) 


(89) 


The  3x3  case  is  valid  for 


•1 


B B(t)  B = B(t) 


8 


8 


3x3 


that  is 

r 


0 

1 

0 


1/3 

0 

1 


r* 

1 

0 

0 


(90) 


= B(t) 


416 


j 


1 


ii 


!i 


.1 


J 


1 


] 


r 


a few  further  observations  will  be  made  with  respect  to  the  derivatives  of  the 
monomial  base.  By  Eq.  (24)  Section  (B) 

= j(j-l)(j-2)  . . . (j-i+1)  (91) 


fi ) 


.(tJ)  = j(i)  tJ-i  = i!  PU-i 


The  equations  of  Eq.  (45) 


/ t 
■3t 

d^/  t 


2 

i.tV'" 


/ 1 


J 


d'^-Vt 


have  as  i,  j th  element  i,  j =0,  1,  2, 

r.  . r . . 


T^(t)  = d*-(tJ) 

dt^ 


For  the  nomomial  base 


'Nm(t)  = (1,  t, 


"n  n'(t)  m(t)  V 


. d-1  by  Eq.  (90) 


we  obtain  elementwise 


r'~-  ■ 1 

! mj  = jmj.i  I 

]"=  0,  1,  2^  . . . d-1 


(98) 


In  order  to  adequately  carry  on  the  direction  here  one  needs  a lot  of  properties 
and  theorems  about  nilpotent  matrix,  full  rank-upper-triangular,  strictly 
upper-triangular  relations  under  full  rank  upper  triangular  similarity  etc. 

These  topics  as  applied  lead  into  spectral  aspects  of  the  matrices  herein 
used  also;  but  will  not  be  pursued  further  here  since  the  prime  purpose  is  to 
obtain  the  time  domain  filter  of  interest  in  trajectory  estimation.  A later 
paper  on  frequency  response  should  be  done  to  tie-in  the  time  domain  with  the 
frequency  domain  aspects  of  optimal  estimation. 

Scott  in  reference  (80)  (1964)  says  that  recently  (M.  L.  Boas  1963)  a formula  for 
the  derivatives  of  Legendre  polynomials  was  derived,  and  that  the  purpose  of 
his  paper  (Scotts)  was  to  obtain  an  analogous  formula  for  the  derivatives  of 
Tchebychef  polynomials  of  the  second  kind  by  making  use  of  properties  of 
Gegenbauer  polynomials. 

Morrison  on  p.  238  of  his  text  (ref  61)  states  that  the  discrete  Legendre 
polynomials  do  not  easily  permit  differentiation,  that  any  attempt  to  dif- 
ferentiat  them  completely  fractures  their  structure  and  all  order  is  there- 
after irretrievable  lost. 


Time  Derivatives  of  the  Exponentially  Weighted  Base  and  Lagrange  Polynomials 
The  exponentially  weighted  base  is  given  by 
-at/2 

./  = 0 <t  (99) 


and  the  derivative  is 


d -at/2  ^ -at/2 

• - = -a  0 -^t+O.  /t 


dt 


\ 


-at/2 

= 0 s'tfV  - all 


(100) 


Let 


then 


< 


t 

e 


4t 


(101) 


(102) 
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and 


^ t =,^  t (V  - bl) 
e ^ e 

/ =<  t^(V  - bl)2  = ^ t^v2 

with  (for  the  4x4  case) 


V = 
e 


-b 

0 

0 

0 


1 

-b 

0 

0 


0 

2 

-b 

0 


0 

0 

3 

-b 


For  the  exponentially  weighted  Poisson  polynomials,  that  is 
<p  = < t 


and 


^Pe  = { P e 

/ Pe  =<P 


-bt 


V 

e pe 


with 


pe 


-b 

0 

0 

0 


1 

-b 

0 

0 


0 

1 

-b 

0 


0 

0 

1 

-b 


Equation  (109)  is  a well  known  inverse  for 


A 

1 

0 

o' 

-1 

A-l 

CM 

1 

1 

A-3 

-A-^ 

0 

A 

1 

0 

0 

A-l 

-a-2 

a-3 

0 

0 

A 

1 

0 

0 

A-' 

-a-2 

0 

0 

0 

A 

0 

0 

0 

A-l 

- 

and  for  \ = -b 


-b 

1 

0 

0 

-1 

-b'l 

-b-2 

-b-3 

1 

1 

0 

-b 

1 

0 

0 

-b-l 

<M 

1 

1 

-b-3 

0 

0 

-b 

1 

0 

0 

-b-i 

-b-2 

0 

w 

0 

0 

-b 

• 

0 

0 

0 

-b-' 

(103) 

(104) 

(105) 


(106) 

(107) 

(108) 

(109) 


(110) 


(111) 


Since  there  are  so  many  versions  of  the  orthogonal  and  orthonomal  polynomials, 
the  classical  and  the  modified  ones  that  only  two  examples  will  be  given  here. 
Consider  Eq.  (54)  which  relates  the  classical  Lagrange  polynomials  to  the 
monomial  base. 
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(112) 


t£ 


<£(t)  = 4.  tD(a")^’^Ri'  = ^ tT 
with  derivative 

=^tV^D(a")A"^Rll 

or  the  derivative  in  the  monomial  base  is  given  by  Eq.  (113)  and  in 
by  Eq.  (112)  in  Eq.  (113) 


^ £T"JVj,D(a")r^Ri( 


n^A-lT 


or 


Powers  of  yield 

'^£  H£  ''t  ^t£ 
and  since 

V‘'  = 0 


then 


(113) 
the^£  base 

(114) 

(1151 

(116) 


vj  = 0 

hence  is  nilpotent. 

Consider  next  the  Lagrange  functions  of  Eq.  (55)  Section  (2) 
-at/2  - 

y£  (t)  = (^  <tD(a")A"^Rt 

^ ^ 5 ^ 


(117) 


(118) 


= / t T ^ 
e ef 


and  by  Eq. 


<;  'f  =<  SV.t  = 


£.T  . V T . 
e ef 


Since  by  Eq.  (105)  is  full  rank,  the  powers  of 

vj  = t"1  T . 

£f  ef  e ef 

is  also  full  rank. 

The  trigonometric  set 

/ f(t)  = (cos  U)  t,  sin  u)  t,  cos  2u)  t,  sin  2u)  t) 
o’  o’  o*  o 


(119) 


(120) 


(121) 


(122) 
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L 


■ luiLiiJii’i  “■' 


has  a velocity  matrix  given  by 


0 u) 


<£(t)  = .(f(t) 


(123) 


(124) 


where 


0 1 
1 0 


(125) 


/ = I 


(126) 


The  base  transition  matrix  is  given  by  Eq.  (10),  Eq.  (11)  and  Eq.  (12) 


<f(t+At)  =<  f(t)  <t>(At) 


(127) 


At  = Iho 


tliat  is  one  step  or 


nAt  = nho 


n steps,  we  can  use  the  discrete  relations  of  Section  (5). 


From  Eq.  (123)  Section  (5)  for  i = o 


< t(n)  = < nT^O)D(ho) 


(128) 


and  for  i = 1 


< t(n+l)  = / nT^(l)T^(p)D(ho) 


(129) 


where  by  Eq.  (38)  Section  (5)  ,for  6x6 


T (1)  = 

u 


111111 
0 1 2 3 4 5 

0 0 1 3 6 10 

0 0 0 1 4 10 

0 0 0 0 1 5 

0 0 0 0 0 1 


(130) 
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Note  that  T^(l)  commutes  with  T^O)D(ho)  hence  Eq.  (129)  becomes 

^ t(n+l)  =^t(n)  T^(l)  = ^ t(n)<l>j.j.  (131) 

with  the  transition  matrix 

*tt  ■ (132) 

In  a similar  manner  for  updating  k points 

<l>^^.(n+k,  n)  = T^(k)  (133) 

Consider  the  case  given  by  Eq.  (99) 

-at/2 

<'t^=(p  <t(n)  (134) 

with 

t = ho(p+n)  = t(n)  (135) 

P = to 
ho 

t(n+l)  = ho(p+n+l)  (135) 

and 

-at(n+l)  ho(p+n)+ho 

^ 

Using  Eq.  (131)  and  Eq.  (136)  in  Eq.  (99)  at  t(n+l) 
ho 


/tg(n+l)  =0  ^ t^(n)  T^(l) 

or 

(137) 

4tg(n+l)  = 4tg(n)'Dg 

(138) 

with 

ho 

4»  = ^ T (1) 

o'"  u 

(139) 

and  for  k data  points  ahead 

l\t  (n+k)  = / t (n)  <l>  (k) 

'e  -^e  e 

(140) 

kho 

2 

♦^(n+k,  n)  = T^(k) 

(141) 

422 


T 


1 


Note  that  the  transition  matrix  of  Eq.  (105) 

=0(V-bI)t  ,^.-bt^Vt 

Eq.  (110)  is  true  because  the  matrices  V and  bl  commute. 
The  transition  matrix  for  the  monomial  base  by  Eq.  (18)  is 
♦ (t,o)  =0‘‘’V^(t)B(l)D(t) 


(142) 


(143) 


For  the  exponentially  weighted  case  into  the  infinite  past  with  zero  measurements 
there  one  should  use  the  front  of  the  span  to  the  back  of  span  indexing  with 
the  back  span  point  going  to  negative  infinity.  By  Eq.  (172)  and  Eq.  (173) 
Section  (5). 


aho(P+l)  -ahoj 

Cr  2 ' 2 


ao 
2 

=e  e 


y/2 


(144) 


and  by  Eq.  (176)  Section  (5) 

/.  ^ YT^(je)T^(p)D(ho) 

For  p data  point  advance  of  the  front  of  the  span,  the  time  becomes  by  Eq. 
(170)  Section  (5) 


and  Eq.  (112)  becomes 

A'x  a.bolg.+p+£)  -ahoy 
V'  2 V'*  2 

The  base  by  Eq.  (113)  becomes 

ao  a 

'2  2 

/ f^(£+M,  Y)  = C-",  9 

and  due  to  the  commutitivi 
particular  case)  one  has 


(145)  ' 


1 - hoy 

(146) 

ao  ^ -jj 

2 I 2 

(147) 

= 0 6 0 

t -M 

'7 

e 4YT^(l)T^(M)T^(P)D(ho) 

(148) 

of  the  T^  (upper  triangular)  matrices 

in  this 

(£,y)  T^(m) 

(149) 

where 


<1)  = (p) 

and  for  one  point  forward  advance  one  has 
♦^(^^+1,  Y)  = e'^T^(l) 
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(150) 


(151) 


and  for  the  3x3  case  the  exponentially  weighted  polynomial  fitting  function 
transition  relation  is 


Section  9 


FITTING  FUNCTIONS  STATE  EQUATIONS 
The  state  equations  for 
x(t)  = <f(t)  ^ 

are  obtained  in  this  section.  The  derivative  of  Eq.  (1)  is 
X = <f  ^ 

where 

or 

i = <f  Vja> 

The  second  derivative  is 

x'  = <{  Vj  a>  a <f  Vj  ,> 

By  Eq.  (5)  we  see  we  need 

<£  =<fv 

and 

f = <^fV  +<|  V 
= <f  [V^  + V] 

The  third  derivative  is 

<^f'=  <f  [V^  + V]  +<^  [2VV  + V] 
or 

<^  f =<l  (V^  + 3VV  + V] 

For  the  many  cases  of  fitting  functions  discussed  in  these  pages 
V = 0 

hence  Eq.  (8)  becomes 

<^1  =<| 

<f  =<( 

=< 

<f‘^  =4  v'* 
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(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

(9) 

(10) 

(11) 


and  for  those  fitting  functions  with  nil-potent  V 
= 0 


The  state  vector  is  for  condition  of  Eq.  (11) 


^ (t) 

5 


F (t))> 


Eq.  (13)  can  also  be  written  as 


x>^  ^ (v'^)2  f(tj> 

.<a  (V'^)‘^"^. 

= T^(a)  f (t)> 

The  derivative  of  the  state  vector  is 

^>s  = f(t)> 

Inverting  Eq.  (13)  and  Eq.  (15) 

»>  = x> 


£<t>=  F^(a)-‘ 

Using  Eq.  (17)  and  Eq.  (18)  in  Eq.  (16) 
x>  = F (t)  x> 

^S  V f V ■^s 


x>  F x> 


= F,(t)  Vj  F;‘(t)  = F^(a)  f;'(.) 


For  the  monomial  base  (polynomials) 


F = S 
X uo 


'0100... 
0 0 1 0 . . . 
0 0 0 1... 
,00  . . 0 . . 


(23) 


For  the  trigonometric  base  of  Eq.  (122)  sec  (8)  and  Eq.  (123)  sec  (8)  for  the 
4x4  case 


F = F V f"^ 

X WWW 


(24) 


or 


F = 

X 


^ ' 2 
3 '3  ^ 


J + ^ I 
3 3 


i J - ? I 
3 3 


--  J + - I 
3 3 


2 /-I  -l\  + 1 / 4J  J\ 

3 \4I  41  / 3 \^-4J  -J  j 


(25) 


(26) 


where  the  nomalization  of  the  W factor  has  been  assumed. 

o 

The  fitting  function  transition  matrix  of  Eq.  (6)  is 


<|(t)  =<(o)  e 


(27) 


or 


4(t)  =4(0)  [I  + V t + V t^  + . . .] 


2! 


(28) 


where 


4-^  (t,o)  = [I  + v^t  +...]  = e 


(29) 


which  is  a Taylor  series  expansion. 


The  state  transition  matrix  by  Eq.  (20)  is 


K(t5>^  = e 


F (a)  v/  F "'(a)  t 

V f V 


■(O^X 


(30) 


or 


x(t)>^  = e 


F (t)  V.  F "^t)  t 

V f V 


x(0)X 


(31) 


with 


\ (t,o)  = e 


F V,F"^t 

V f V 
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(32) 


- x 


r 1 

1 

s 

4>  = F <D.  F"^ 

X V f V 

(33) 

/ ■ 

(- 

Consider  the  states  given  by  Eq.  (14) 

) 

• 

x(t)>^  = T^(a)  f(t)> 

(34) 

and  at  t + At 

J : 

£ 

x(t  + At)>g  = T^(a)  f(t  + At)> 

(35) 

if 

L 

<f(t)  =<f(t)V^ 

(36) 

, 

for  constant  matrices 

<$(t)  =<f(t)e'^f^'''''°^ 

(37) 

or 

i ; 

<f(t)  =<f(to)lI  + V (t-to)  + vl  (t-to)^  + . . .] 

(38) 

' 

* ^ 2! 

) ' 

; ( 

if  we  set 

i 

At  = t - to 

(39) 

} : 

or 

t = to  + At 

(40) 

In  terms  of  the  transition  matrix 

<f(t)  =<:J(to)  <l>^(t,to) 

(41) 

and  Eq.  (35)  becomes 

x(to  + At)>^  = T^(a)  <l>^(t  + At,t^)  f (t^J> 

(42) 

or  for  any  t 

x(t  + At)>^  = T^(a)  <t>^'t  + At,t)  f(t)> 

(43) 

1 

Inverting  Eq.  (34) 

f(t)>  = ria)  x(t)>^ 

(44) 

i 

and  using  Eq.  (44)  in  Eq.  (43) 

[ 

i 

x(t  + At)>^  = 1^(3)  <|.J(t  + At,t)  T’^a)  x(t)>g 

(45) 

> 

1 

or 

x(t  + At)^g  = 4>^(t  + At,t)  x(t)^^ 

(46) 

.1 
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} 

, 

fcL- 

. i - t 

— 

with  state  transition  matrix  given  by 

<t>^(t  + At.t)  = T^(a)  <t.J(t  + At,t)  T^^a)  (47) 

For  the  3x3  case  the  matrix  V is  by  Eq.  (22)  for  the  second  degree  polynomial 
monomial  base  as 


T 

= 


/O  0 0\ 

1 0 0 

\0  2 0/ 


(48) 


and 


<a  = (a^.  a^,  a^  /O 


0 

1 0 
0 2 


0\  = (a^,  2a2,  0 ) 
0/ 


and 


<^a(v''’)^  = (aj,  2a2,  0)  /O  0 0\  = (2a  , 0,0) 


hence 


/O  0 0\ 

1 0 0 

\0  2 0/ 


(49) 


(50) 


T (a)  = 

V 


"O  ^ ^2 

2a2  0 

2a2  0 0 


(51) 


One  can  compute  T (a)  and  then  compute  by  Eq.  (47)  to  obtain  4>  (t  + At,t); 
however,  this  route  is  tedious.  By  Eq.  (19)  ^ 

x(t)>^  = Fv(t)  x(t)>^  (52) 

and  for  the  polynomial  case  considered  by  Eq.  (220)  sec  ( 2 ) 

(53) 


„-l. 


S = T (a)  T *(a)  = /O  1 O' 
uo  v fv  ' 


(•  • ’) 

\0  0 0/ 


Also  Eq.  (52)  has  solution 


■1. 


F V^F  t 

^(t)>g  = e ~ 


where 


F Vj'^t  V,t 

I,  = e ^ ^ = F e ^ F'^ 

X V V 


with 


e = *. 


(54) 


(55) 


(56) 
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The  state  transition  matrix  is  given  by  Eq.  (160)  sec  (4)  as 


'1  At  ^ 
2! 

<l>  (t  + At,t)  =01  At 
^ 0 0 1 


The  base  transition  matrix  is 

<l>  = I + V At  + 

I I * 2 ! 


/ 1 At  At 

4>  = I 0 1 2At 

\0  0 1 > 


Note  the  difference  in  the  two  transition  matrices  for  this  case 
As  an  example  consider  the  monomial  base  of  degree  2 
<t  = (1,  t,  t^) 


<t  = t 


<t(t)  = t(0)e^  = <t(o)  <Dt(t,o) 


4.  (t,o)  = (I  + V t + V ^ ) 
L L L 21 


/I  t t2\ 

4>  (t,o)  = 0 1 2t  = B(t) 

\0  0 1 / 

Where  B(t)  implies  the  binomial  coefficient  matrix.  By  Eq.  (53) 
x(t)>  = S x(t)>  = T (a)  T“^(a)  x(t)> 

^S  UO  ■'s  V tv  '^S 


x(t)'>  = <t>  (t,o)  x(o)^ 


<1>  (t.o)  = 6 


St  „ , 

= I + S t + t^ 


Consider  as  an  example  now  the  exponentially  weighted  polynomial  case.  The  base 
transition  matrix  is  given  by  Eq.  (107)  sec  (2)  as 


4> 


_ g ho/2  .j. 

u 


for  one-step  advance. 

By  Eq.  (105)  sec  (2)  for  the  3x3  case 


V = V - bl 
e t 


and  the  states  by  Eq.  20 


x(tj>^  = F^(a)(V^.  - bI]F;;^(a)  x(t|>g 
By  Eq.  (110)  and  Eq.  (Ill)  sec  ( ) 

= F^(a)  <}>^(t,o)F'^(a) 

♦ (t,o)  = F (a)  D"\t)  B(l)  D(t)  F'^a) 


The  matrix 


,(a)  = <a 


-bao  + aj 


b a - 2ba, 
o ] 


-ba^  + 2a 


B^a. 


2 

4ba. 


Eq.  (82)  is  seen  to  be  quite  difficult  to  invert. 


(77) 


(78) 

(79) 

(80) 
(81) 


(82) 


SECTION  10 

GENERAL  RELATIONS  FOR  PARAMETER  ESTIMATION 

This  section  considers  discreet  estimation  for  the  observational  system 

z(t)(m^  = y(t)(m)’j  + u(t)(m^j  (1) 

z(t)(m^  is  an  m-dimensional  time  varying  measurement  vector,  y(t)(m^  is 
the  signal  and  u(t)(m  is  the  additive  measurement  noise;  the  sequence 
j=o,...j  where  j will  represent  a countably  infinite  number  and  is 
to  be  thought  of  as  the  discreet  population  space;  or  the  j experiment. 

The  induces  to  be  tagged  onto  t (the  sample  space  variable)  will  correspond 
to  the  double  induces  for  a fixed  span,  forward,  central  and  backward.  The 
following  sections  will  consider  the  recursive  cases  corresponding  to 
infinite  memory  filters  and  fixed  memory  of  moving  fixed  span  filters. 

The  signal  sequence  y(t)(m ^ will  be  considered  for  a number  of 
cases  som  of  which  are  stochastic,  some  deterministic. 

For  example  if 


y(t)(m|j  = H(t)  x(t)(p)^ 


then  the  p-dimensional  state  vector  is  stochastic,  and  may  be  the  output 
of  a linear  dynamical  system.  The  parameter  estimation  problem  for  the 
special  case  where  the  degree  of  the  polynomial  or  the  nature  of  the  fitting 
functions  is  known,  that  is 

x(t)(p^=  B f(t)(d^  (3) 

pxd 

where  the  signal  is  exactly  expressible  as  a linear  comination  of  d 
fitting- functions  (deterministic  with  no  - approximations)  is  considered 
first.  In  this  section  the  scalar  measurement  case  will  be  considered. 


that  is  m=l 


z(t)j  =^)h(t)  B f(t)(d^ 
pxd 


with  the  additional  constraint 


^)h(t)  = (1,0,0, ...0) 
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In  a later  section  the  case  will  be  considered  where  the  measurement 
is  scalar  position  x(t)  and  scalar  velocity  x(t),  that  is 

1 0 0 ...  0 


H(t)  = 


and 


2xp 

y(2^  = 


0 10. 


(6) 


(7) 


The  case  for  this  section  thus  is 


Zj(t)  = Xj(t)  + u^.(t) 


(8) 


oi- 


Zj(t)  = d)f  (t)  a(d  + Uj(t) 


(9) 


that  is 


x(t)  = f (t)  a + f,(t)  a + . . . + f (t)  a, 

o o 1 1 a-1  d-1 


(10) 


at  the  end  of  this  section  some  examples  will  be  given  for  the  indicesion 
time 


^d)f(i,n),  ■^f(k,m),  ^f,  j. 


(11) 


but  for  the  most  part  the  number  of  observations  will  be  k,  that  is  a k- 
vector  of  measurements 


"k-l^j 


■^k)jZ  = (z^,  z^,  . 

and  by  Eq  (9)  for  the  sample  column  vector 

z(k  F a(d^  + v(k^j  = F a(d^.  = z(k^j 
kxd  kxd 


(12) 


(13) 


where 


F = 
kxd 


^)f(o)  ; 


(14) 


and  the  vector  a(d^  is  the  estimate  of  the  parameter  vector  a/^  for  the 
test.  Note  that  the  vector  a^ is  assumed  the  same  each  test. 

For  d < k and  the  rank  of  F equal  to  d,  that  is 


p F = d 
dxk 


(15) 
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.1 


then  there  exists  many  rank  d matrices  w of  size  dxk  such  that  Eq  (13) 
becomes 


where 


w z . = 0/ 

: / 


L = wF 
o 

dxd  dxd 

is  square,  full  rank  (hence  invertable)  and  a constant  matrix. 
Multiplying  Eq  (16)  by  L~^ 


w z).  = a^  + w v)  = a ^ 


Using  Eq  (19)  in  Eq  (13) 

z^=  F(wF)  ^ wz^  + z ^ (2( 

or 

z;.  = + z^.  (2: 

where 

z ^ = F(wF)  ^ w z^  (2: 

Note  that  the  matrix  of  Eq  (22)  is  a projector  (idempotent  index  2) 

for 

[F(wF)“^  w]CF(wF)"^  w]  = F(wF)"^  w (2c 

also  by  Eq  (20)  solving  for  z 


z ^ = [I  - F(wF)~^  w]  z^ 


where  the  matrix  I - F(wF)~^  w is  also  idempotent  index  2. 
If  the  matrix  of  Eq  (18)  equals  I,  that  is 

Fw  = L = I 
o 

then  the  projector  of  Eq  (22)  and  Eq  (23)  becomes 

P„  = Fw  = 

Fw  Fw 


and  the  oblique  compliment  projector  of  Eq  (24)  becomes 

- -2 
p = I _ p = 

Fw  , , Fw  Fw 
DX] 


i 


and 


"fv  "fw  = ° 


(28) 


By  Eq  (22)  and  Eq  (25) 

z^.  = Fw  tS.  - P„  ZA 
• s Fw  A] 

and  by  Eq  (24) 

'Is  = ^Fw  '>j 

By  Eq  (19)  and  Eq  (25) 

« z^j  = a^+  w = a^j 

For  the  special  case  of  the  weighting  matrix  w equal  to  the  psuedo- 
inverse  of  F,  one  obtains  the  unweighted  solution,  that  is 


(29) 


(30) 


(31) 


* T -1  T 
w = F = (F  F)  F 


dxk  dxk 
and  by  Eq  (19) 


^ = F*  z}j  = (f’^F)"^  f"^  z^ 


(32) 


(33) 


(34) 


(35) 


which  is  the  standard  "normal"  equation  for  the  parameter. 

The  estimate  of  the  measurement  vector  is  by  Eq  (29) 

PpF*  ^ 

where  the  symmetric  projector 

= (FF*)  = F(f'^F)"^  F 

and  the  vector  z^ of  Eq  (30)  is  orthogonal  to  the  vector  of  Eq  (34).  The 
special  cases  of  Eq  (32)  will  be  considered  after  a number  of  other  results 
are  obtained. 

For  the  weighting  case  using  Eq  (25)  in  Eq  (19) 

w z^j  = ^36) 

Partitioning  the  weighting  matrix  w into  its  row  space  dxk 

. . (37) 


w = [w(d^^,  w(d^2  • 
dxk 

and  using  Eq  (37)  in  Eq  (36) 

a(d) . = w(d\  z + . 
» 9 /o  o 


. + w(d)^_^  z^_^ 


(38) 
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or  the  d-dimensional  parameter  vector  is  a linear  combination  of  the 
weighting  vectors.  Note  that  the  weighting  vectors  are  to  be  determined 
in  some  optimal  sense  such  that  they  are  the  same  each  experiment,  that  is 
they  are  not  to  be  a function  of  j in  some  "future  sequence";  but  will  be 
a function  of  some  "£  prior"  past  sequence  of  tests. 

The  error  in  the  estimate  of  the  function-parameter  by  Eq  (36)  is 


> - a>.  = a>.  = -w  z). 


] 


(39) 


If  the  population  sequence  j of  Eq  (13)  is  packaged  as  a row  of 
column  vectors  we  have 


[z(k^^,  z(k^2  • • • z(l^j]  = Z = F a((^^)  1 + V (40) 

kxj  kxd  kxj 

Z = F A + Z . (41) 

kxj  kxd  dxj 

Consider  next  the  package  of  measurement  noise  vectors  of  Eq  (40) 

V = [v(k)^,  . . . v(k^j]  = v(l) 


) v(2) 


(42) 


I 


) v(k-l) 


Equation  (42)  consideres  j column  vectors  in  k-space  and  k row  vectors 
in  j -space. 

The  mean  (or  arithmetic  average)  of  the  j column  vectors  in  k space 
of  Eq  (42)  is 


V 1 (j^  = v(l^^  + . . . v(l^^  = p (1^^ 

kxj  j 

max 


(43) 


where 


*(j)  = l(j  > 


max 


i(i> 


(44) 


If  the  k-dimensional  mean  vector  of  Eq  (43)  is  the  zero  vector  in  k-space 
then  o(k  is  the  point  of  symmetry  with  respect  to  the  sequence  of  j vector. 
The  j-dimensional  row  space  interpretation  of  Eq  (43)  is 

* 

(45) 


V 1 


<i>' 


^v(o) 

^)v(l) 


i‘<]> 


j^)v(k-l) 
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t 


V 1 (j  ) = [y(o),  w(2)  . . . W(k-l)]  (4e 

V l*(j>  = u(k)^  (4’ 

By  Eq  (45)  we  see  a column  vector  of  inner-products  and  if  the  y(k^^ 
vector  is  zero  in  k-space,  then  all  k of  the  row  vectors  in  j-space  are 
perpendicular  to  the  1 (j  > . 

The  mean  of  the  k row  vectors  in  j-space  is 

^k)l*  V =fk)l*  ^)v(o)  = ^)in  (4( 

kxj  1? 


^)v(k-^ 

with  a corresponding  inner-product  interpretation. 
In  j-space  for  the  k vector 


/j)v(k)  =^)m^ 


or  package-wise 

V = l(k><(j)m^  + 
kxj 

By  Eq  (48)  in  Eq  (50) 

V = Kk")  ^k)l*  V + V 
kxj  / ' 


= (I  - l(J^<[k)l  )V  (52) 

kxj 

which  is  projection  in  k-space. 

The  most  useful  projection  notion  is  in  j-space.  Consider  the  j column 
vectors  in  k-dimensional  space  of  Eq  (42),  which  by  vector  addition  may  be 
decomposed  as 


v(k)^  = u(k)^  + v(k)  . 


or  package-wise 

V = u(k^  ^j)  1 + V 
kxj  kxj 

Using  Eq  (47)  in  Eq  (54) 
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V = V 1 (j^  ^j)l  + V 
kxj  kxj 


(55) 


or 


V = V (I  - 1 (j>^j)  1) 
kxj  jxj 


(56) 


or 


V = V P 


11 


(57) 


also  by  Eq  (55) 

V = V + V 


where 


V = V P 


11 


(58) 

(59) 


and  the  rank-one  projector  in  j-space  is 


p,  = 


(60) 


The  vector  partitioned  into  the  j-dimensional  row  vectors  are 


^•)v(o) 

l^)v(o) 

(j)v(o)  ! 

(j)v(l) 

• 

• 

|■^)v(l) 

= 1 
i 

1 * 

+ 

^j)v(l) 

• 

• 

!/  ■ 

jy  )v(k-l) 

‘ 

1 ^ 

i^)v(k-l) 

• 

)v(k-l) 

! 

■ 

— — 

(61) 


11 


11 


FIG  (1)  ORTHOGONAL  PROJECTION  OF  NOISE  VECTORS. 


Thus  we  see  if  one  has  a zero  mean  noise  case  that  all  vectors  are 
perpendicular  to  the^j  )1  vector  as  shown  in  Fig  (2) 
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subspace 


to 


FIG  (2)  VECTORS  PERPENDICULAR  TO 


If  all  k-vectors  in  j -space  are  mutually  perpendicular  one  has  a 
diagonal  matrix 


We  can  now  define  the  variance  matrix  of  the  measurement  noise  as 


, ~T  ~ 'v 

R = lim  'V  V 1 ! 

max  J 


since 


(v(k)^,.  . . v(k^j  (k)v  f v(k^^'Ck)v 


3 = 1 


The  serial  time  correlations  are  given  by 


r(o,o) 


r(o,l)  . . . r(o,k  ) ' 


R = . 


r(k“^,l)  r(k"^,k“^) 

The  standard  definition  of  the  noise  variance  matrix  of  Eq  (63)  is 


R = C iCvCk).  - C^R)v  - ^k)p  ]\ 

I.  3 j V ' 


Return  now  to  Eq  (19)  where  we  have  placed  the  constraint  on 


w z(R^j  = o((^ 


for  all  tests  j . 

We  can  multiply  Eq  (40)  by  w 

A 

wZ  = a((^  ^j)l  + wV  = A 


where 


Define 


wZ  = o 
dxj 


A = a(d^  )1 
dxj 


then  by  Eq  (68) 


A - A = A = -wV 


= -w[p(k}  ^ j )1  + V] 
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Z = FA  + V = FwZ  + Z 


Z = (I  - Fw)Z  = Z 
Fw 

kxj  kxk  kxk  kxj 


and  by  Eq  (72) 


Z = FwZ  = Pj^Z  (7u; 

The  variance  of  the  estimate  of  the  parameters  can  now  be  defined  by 
Eq  (71)  as  

i—  = lim  A 1 

’^aa  

dxd  j-x»  ^max 


i—  = lim 
^aa 


. -^max 


a(d^.  K d)a  1 
^ j 

max 


I 

= 5(a(d^^d 


where  C is  the  standard  expected-value  operator.  In  terms  of  R and  w 
Eq  (75)  is 

I—  = lim  (wwV) 


t~-  = w[p(k)  'Ck)y  + R]  w^ 


lim  Vv'^  = lim  (u(k><i)l  - V)(l(i><k)u  - V^) 

^max 

in  ni3X 


= ij(k}  ^k)y  + R . (79) 

If  the  noise  mean  is  not  known  it  is  clear  by  Eq  (68),  Eq  (71)  and  Eq  (79) 
one  can  not  separate  the  constant  noise  mean  vector  from  the  constant 
parameter  fitting  the  function  vector.  It  will  be  assumed  hence  forth  that 
the  measurement  noise  mean  is  known  and  can  be  subtracted  out.  In  the  real 
world  applications  one  may  use  Friedlands  bias  estimation  technique  to 
estimate  the  bias  in  the  measurement  noise  and  in  the  plant  noise  or  function 
fitting  mis-modelling. 


. 


Multiplying  Eq  (72)  on  the  left  by  Pp^ 


since 


P„  z = P_  V = Z 
Fw  Fw 


P„  FA  = o 
Fw 


since  by  Eq  (28) 


P^  F = o . 
Fw 


(80) 


(81) 


(82) 


The  variance  of  the  estimate  of  the  measurements  by  Eq  (80)  can  be 
written  as 


i—  = lim  (Z  1 ) 

33  • • 


(83) 


^max 


or 


max 

V 


ha  = W <'k)p  + R) 


' ^Fw  ^ ’^Fw 


Fw 


(84) 


since  we  assume  y(k^y^  k)ii  ai4d  the  oblique  projector  is  known;  or  equivalently 
assume  the  noise  is  zero  mean. 

The  results  of  Eq  (63),  Eq  (79)  and  Eq  (84)  can  be  compactly  written. 


V 

1 

I : 

kxj 

kxk  ^ 

z 

* 

^Fw  1 

kxj 

j 

A 

-w 

dx  j 

j 

- 

(85) 


which  is  a function  of  the  single  independent  matrix  V (ignoring  u(k^y) 
Transposing  Eq  (85) 


(v\z\a^)  = v^d,  P^,  -W^ 
and  foxifling  the  outer  product 


(86) 
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V I (v'^,z’‘,a’^)  = I 


V v^d.  . V) 


rj 


■■  “T  ~ ~T  ” ~T 

V V V Z V A 

- ~T  ->  -"p  - ~.p 

Z V Z Z Z A 

~ -T  ~ ~'p  ~ ~IT1 

A V A Z A A 


- -T 

V V 

~ "T  ~ T 

V V P * 

Fw 

“ "’T  T 
-V  V w 

~ ~ ~T 

P„  V v" 

Fw 

P„  V v'  P„ 

Fw  Fw 

-Pj,  R w' 
Fw 

- -xp 

-w  V V 

••  -T  ~ T 
-w  V V*  P„ 

Fw 

- -.p 

+ w V V 

and  at  the  variance-covariance  level 


t- 

I 

^VZ 

^va 

1 

t- 

^zv 

^ZZ 

^za 

t-- 

■ ^av 

^az 

^aa 

1 

T 

R 

R Pt. 

-R  w 

Fw 

P R 

- - P 

P„  R P^ 

-Pt^  r 

Fw 

Fw  Fw 

Fw 

- P 

-w  R 

-w  R P„ 

Fw 

w R w 

Note  that  Eq  (85)  is  a j x (2k  + d)  matrix  and  a matrix  of  variance 
of  Eq  (87)  and  Eq  (88)  is  a (2k  + d)  square  matrix  with  rank  given  by  Eq  (87) 
as  k when  R is  full  rank  equal  to  k. 

For  the  special  case  where  the  parameter  vector  has  dimension  one,  that 
is  by  Eq  (10)  and  Eq  (8) 

z.(k)  = a + v,(k)  (90) 

3 o 3 


or  for  d=l  and  w a row  vector 


w = <^)  w 
Ixd 


and  by  Eq  (14) 


F = 
kxl 


= Kk} 


• I 


t = I R 


R P/ 
Iw 


(2k+d)(2k+d) 


I P,  R 
Iw 


~ *■  T 

P,  R P/ 
Iw  Iw 


- w R 


~ T 

- w R P/ 
Iw 


By  Eq  (40) 


Z = FA  + V = FA  + Z 


or 


V = -F(A)  + Z 


= (I,-F) 


and  transposing 


v"  = (z",  7) 


The  product 


L J 


V = (I,  -F)  it—  t — 


-R  w 

J-W 

w R w 


^ I 


which  interestingly  "looks-like”  an  algebraic  Riccatti  equation  in  F. 

By  Eq  (89)  we  see  that  the  variances  and  covariances  of  the 
measurements  and  the  parameters  are  dependent  on  the  measurement  noise 
variance  matrix  R,  the  fitting  function  matrix  F and  the  weighting  matrix 
w.  If  one  selects  the  fitting  functions  to  be  used,  for  example  polynomials 
of  a known  degree,  trigonametric  etc,  then  F is  determined.  The  matrix  R is 
a characteristic  of  the  measuring  instrument  and  in  the  real  world  one 
attempts  to  estimate  what  it  is ; however  for  the  purpose  here  it  will  be 
assumed  known.  The  only  other  matrix  of  concern  is  the  optimal  weighting 
matrix  w.  A number  of  cases  can  be  considered:  let  w imply  weighted, 
w (compliment  meaning  no  weight)  and  like-wise  for  constrained  parameter  c 
and  unconstrained  parameter  c,  we  thus  have  the  four  cases 


(w,w) 


where  the  cases  are 


|c  w 
iC  w 


cw 
cw  ; 


(99) 


1. 

c w - unconstrained 

- unweighted 

2. 

c w - unconstrained 

- weighted 

3. 

c w - constrained 

- unweighted 

4. 

c w - constrained 

- weighted 

The  first  case  is  considered  first  in  the  following  section. 


Section  11  UNCONSTRAINED  UNWEIGHTED  PARAMETER  ESTIMATION 


The  unweighted  parameter  estimation  cases  considered  in  this  section  use 
the  W matrix  to  be  the  psuedo- Inverse  of  F given  by  Eq.  (32)  sec  dO)  as 


T -1  T 

W - F*  - (F  F)  F 


with  the  measurement  estimates  given  by  Eq.  (34)  sec  (ICj)  as 


5 (^  = FF*  z (^ 

j j 


The  parameter  estimate  is  given  by  Eq.  (33)  sec  00  as 

„T„,-1  T 


£ - F*  ^ - (f’^F)"^  f’’  z (^ 

J ^ J 


The  projections  and  compliment  projectors  are 

,T 


p « ff*  = p' 
fw  ff* 


and 


p = I - ff* 
FW 


P = p'^ 
ff*  ff* 


The  matrix  of  variances  for  this  case  becomes  by  Eq.  (89)  sec  OCi) 


E = 

R 

S 

2-- 

vz 

2,- 

^vi 

2~~ 

(2k+d) (2k+d) 

zv 

zz 

za 

S 

2~- 

2~~ 

av 

az 

aa 

R 


RP 


FF* 


-RF* 


^FF*^ 


'T 

P RP 

FF*  ^FF* 


-Pff*^” 


-F*R 


~T 

-F*RP 

FF* 


F*RF* 


We  shall  look  at  the  "normal  equations"  of  Eq.  (3)  for  some  "often  used" 
cases. 


Case  1.  Back  to  front  (forward)  Polynomials 

The  fitting  functions  are  the  power  series  or  polynomials,  that  is 
<f(t)  - (1.  t,  t^.  ... 

and  for  the  Indexing  given  by  Eq.  (123)  sec  (5),  we  have 


(1) 


(2) 


(3) 


(4) 


(5) 


(6) 


(7) 


(8) 


<t(l,n)  - <n  T^(i)T^($)D(ho) 


(9) 


If  we  assume  batch-processing  (not  recursive  spans)  set  1 ~ 0 In  Eq.  (128) 
sec  (5)  and  Eq.  (9)  becomes  a function  of  n > 0,  L,  2 . . . N 

<t(n)  -<n  T^(B)D(ho) 

and  for  the  second  degree  polynomial  case  by  Eq.  (34)  sec  (5) 


Ty(B)  = 0 


B B 


where  by  Eq.  (16)  sec  (5) 
B = to/ho. 

By  Eq.  (34)  sec  (5) 


D(ho) 


ho  0 
0 h^o 


The  F matrix  of  Eq.  (14)  sec  (10  becomes  that  of  Eq.  (124)  sec  (5) 

<t(o)~ 

<f(l) 


T = 
(N+l)x3 


N T (B)D(ho) 
(N+l)x3  ^ 


<t(N) 


The  transpose  of  Eq.  (14)  Is 

t"^  * D(ho)  t'^(B) 

3x(N+l)  3x3  “ 3x(N+l) 

T 

The  matrix  N Is  given  by  Eq.  (50)  sec  (5)  as 


1 1 ...  1 

2 3 ...  N 

2 2 2 

r 3^... 


N « 0 

3x(N+l)  - 


The  Inverse  of  the  discrete  metric-matrix  required  In  Eq.  (1)  Is 


(tTt)-I  . Ii)(ho)Tj^(B)N^NT^(6)D(ho)  ] "^ 
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(18) 


(T^T)"^  - D"^(ho)T“^(6)(N'’^N)"V'^(6)D"^(ho) 

The  inverse  T^^(&)  is  given  by  Eq.  (149)  sec  (A)  as 


T~^(6) 

u 


1 -6  3 

0 1 -23 

0 0 1 


The  inverse  (N^N)  ^ is  given  by  Eq.  (118)  sec  (5)  as 


3N^+3N+2 

-(12N+6) 

10 

T -1 
(N^N)  ^ = 

3 

(W-1)  (N+2)  (N+3) 

-(12N+6) 

4(2N+l)(8N-3) 

N(N-l) 

-60 

(N-1) 

10 

-60 

60 

(N-1) 

N(N-l) 

By  Eq.  (3),  Eq.  (15)  and  Eq.  (18) 

a = D~^(ho)T"^(3)(N'’-N)"^N^ 

T 

Consider  the  term  N zN  of  Eq.  (21) 

Fl  <$  1 "I  /zo  \ 

n'^z  (N^  =0  <^)  c (^z 

^ 0 <^)  _ 

zo  -K^l 

^^C^  2(^ 

where  the  counting  number  vector  is 
<N)  c = (1,  2,  3,  4,  ...  N) 
and  the  vector  of  squares  of  the  counting  numbers  is 
<^)  c^  = (1,  2^,  3^,  ...  N^) 

and 

<^)  1 = (1,  1,  1,  ...  1) 

Eq.  (23)  can  be  written  as 


N 


I 

z(l) 

i-1 

N 

1 

iz(i) 

1-1 

N 

z 

l^z(i) 

i-1 

(27) 


Thus  Eq.  (27)  used  with  Eq.  (20)  in  Eq.  (21)  for  arbitrary  t^  and  constant 
data  sampling  interval  h yields  the  estimate  of  the  parameters. 

- o --  N 


XT 


hou 


3N  +3N+2 


-(12N+6) 


10 


-(12W+6)  10 

4(2W-l)(8N-3)  -60 

N(N-l)  N-1 

-60  60 

N-1  N(N-l) 


ZO+  Y z(i) 

i-1 

N 

2 iz(i) 

1=1 

I ±h(i) 

i=l 


(28) 


where 


3a 


(N^-l)  (N+2)  (N+3) 


(29) 


and 


(30) 


The  variance  of  the  estimate  of  the  parameters  is  given  by  Eq.  (7)  as 


aa 


T*RT* 


(31) 


The  psuedo-inverse  of  T given  by  Eq.  (14)  is 

T*  = D"^(ho)T"^(e)N*  (32) 

3x(N+l)  “ 3xN+l 

and  the  transpose  is 

T*'^  = N*'^T"^(8)D"^(ho)  (33) 


Using  Eq.  (32)  and  (33)  in  Eq.  (31) 

X aa  » D~^(ho)T"^(6)N*N*'^T"'^(8)D"^(ho)  (34) 

3x3  “ “ 

For  the  analytically  tractable  special  case  of  a scalar  matrix  R,  that  is 
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r 


R 

(N+1) (N+1) 
Eq.  (34)  becomes 


a I 

V 


^ aa 
3x3 


D~^(ho)T“^(6) (N^N)"^T  ^(B)D"^(ho) 


(35) 


(36) 


since 


(N*N*'^)  = (n'^N)*  = (n'^N)  ^ 
3x3 


or  by  Eq.  (20)  in  Eq.  (36) 


(37) 


aa  "''hou 


Z 


3x3 


3N^+3N+2 

-(12N+6) 

10 

-(12N+6) 

4(2N+l)(8N-3) 

N(N-l) 

-60 

N-1 

-T 

^hou 

(38) 

10 

-60 

60 

N-1 

N(N-l) 

where  A and  are  given  by  Eq.  (29)  and  (30). 

Case  2.  Mid-point  Polynomials 

The  mid-point  case  for  the  monomial  base  of  polynomials  is  given  by  Eq.  (128) 
of  sec  (5)  for  k = 0 


<J(in)  =<3tt  T^(6)D(ho) 

and  the  package  of  fitting  fvmctions  at  the  m indices  by  Eq.  (129)  as 

<^(-M) 

<t(-M+l) 


(39) 


T 

(N+l)x3 


<^(-1) 

<(1) 


<t(f0 


M T (B)D(ho) 


(N+l)x3 


(40) 


where  N • 2M 
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The  matrix  Is  given  by  Eq.  (66)  of  sec  (5)  as 


3x(H+l) 


1 1 ...  1 1 1 1 

-M  -(M-1)  ...  -2  -1  0 1 

4 10  1 


2 2 
,M  (M-1) 


1 

2 

4 


1 

M 

M^J 


T -1 

The  matrix  (T  T)  for  this  case  is  by  Eq.  (18) 

T -1  -1  T -1  -1 

(T  T)  - T.  (M  M)  T. 

hou  hou 

T -1 

The  matrix  (M  M)  is  given  by  Eq.  (120)  sec  (5)  as 


T -1 
(M^M)  ^ 


3(3N  +6N-4) 
4(N^-1) (N+3) 

0 

-15 


0 

12 


-15 


N(IH-l)(N+2) 

0 


(N  -1) (N+3) 


(N^-1) (N+3) 

0 

180 

N(N+2)(N+3) (N^-1) 


The  estimate  of  the  parameters  by  Eq.  (3),  Eq.  (40)  and  Eq.  (42)  is 


where 


^+l)  z = (z(-M),  ...  z(-l),  z(0),  z(l)  ...  z(M)) 


The  variance  of  the  estimate  of  the  parameters  is  given  by  analogy  with 
Eq.  (38)  for  the  scalar  matrix  noise  case  as 


I- 

aa 


XT 


-1 

hou 


(N  -1)  (lH-3) 
Case  3.  Front  to  Back  Polynomials 


3(3N  +6N-4) 
4(N^-l)(N+3) 

0 

-15 


0 

12 


-15 


N(N+l)(lW-2) 

0 


(N  -1) (N+3) 
0 

180 


N(N+2)(N+3)(N  -1) 


(41) 


(42) 


(43) 


(44) 


(45) 


C~^ 

hou 


(46) 


The  indexing  from  the  most  recent  "real  time"  data  point  to  the  back  as  given 
by  Eq.  (135)  sec  (5)  is 


F - T - r T (8)D(ho) 
(N+l)x3  (N+l)x3 


(47) 


and 
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T T T 

F - D(ho)T^(e)r 


The  inverse  of  the  discrete-metric  matrix  is 


The  "number- theoretic  discrete  metric  matrix"  inverse  by  Eq.  (279)  sec  (5)  is 


(r^D-^ 


3N^+3N+2 

12N+6 

10 

12N+6 

4(2N+1) (8N-3) 

60 

N(N-l) 

N-1 

10 

60 

60 

N-1 

N(N-l) 

where 


(N+1) (N+2) (N+3)  ' 

Note  that  Eq.  (50)  is  the  same  as  Eq.  (20)  except  for  the  alternating  sign 
changes  in  the  diagonals  from  right  to  left.  Hence,  the  estimate  of  a^and 
the  variance  will  be  the  same  except  for  this  sign  changes  and  will  not  be 
presented  here. 

Case  4.  Discrete  Orthogonal  Polynomials  Back  to  Front 

This  case  is  the  result  of  a Gram-Schmidt  process  on  the  case  1 equations, 
that  is  by  Eq.  (21) 

a (3>  = D“^(ho)T~^(6)  (n’^N)"^  N^  z ( 

By  Eq.  (207)  sec  (5) 

G(N)  = N Bg(N)  ( 

(N+l)x3  (N+l)x3  3x3 

where  by  Eq.  (200)  sec  (5) 

■ Ti  1 1 1 


- 0 -if 


6 

N(N-l) 


T 1 

The  Inverse  discreet  metric  (N  N)  is  given  by  Eq.  (223)  as 


(n'^’n)"^ 


-1  T 
B M B 
g gg  g 


where  by  Eq.  (210)  sec  (5) 
Mgg  - [g^(N)  G(N)]-^ 


or  elementwise 


M 


-1 

gg 


1 

I«-l 

0 

0 


By  Eq.  (53) 

N = G(N)B 


-1 

g 


0 

3N 


(N+1) (N+2) 
0 


0 

5N(N-1) 


(N+1) (N+2) (N+3) 


and 


T -T  T 
N = B G (N) 
g 


The  psuedo-inverse  of  N is 
T -1  T 

N*  = (N  N)  N 
3x(N+l) 


and  using  Eq.  (59)  and  Eq.  (55)  in  Eq.  (60) 

(N^N)"^  n"^  = B (b"^G^(N)) 

g gg  g g 

or 

(n'^N)"^  n'*^  = Bg  (g'‘^(N)G(N))"^  g'^(N) 

or  by  Eq.  (53) 

T — 1 T —ITT 

(N  N)  N = B M B N 
g gg  g 

Using  Eq.  (63)  in  Eq.  (52) 

a = D-^ho)T;;\6)BgM'jBy  z 

Multiply  Eq.  (64)  on  the  left  by  D(ho)  T (B)B“^  to  obtain 

u g 

B-\(6)D(ho)a>=  aS>=  M-Vn^5> 


or 
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(N+1) (N+2) 


5N(N-1) 

(N+1) (N+2) (N+3) 


where 


g‘2>  .0  1 - I 


1 - — 1 - -—  + 


-I 

12  . 12 


N N(N-l)  •• 


For  polynomials  of  degree  higher  than  the  second  degree,  the  elements  of  G 
are  given  by  Eq.  (209)  sec  (5). 

The  variance  is  given  by  Eq.  (55)  in  Eq.  (36)  as 

Z--  = 

aa  u g gg  g u 


T^(6)D(ho)2:--D(ho)TT(6)  = 


where  T®-  is  a congruent  transformation  on  the  variance;  hence  is  a base  change 
and 

Vi.  = B (75) 

^aa  g gg  g ^ ' 

or  in  open  form 


1 -A- 

N " N-1 
° N(N-l) 


3N 

(N+1) (N+2) 


X (76) 


5N(N-1) 

(N+1) (N+2) (N+3) 


6 6 
N-1  N(N-l) 


Case  5.  Midpoint  Discrete  Orthogonal  Polynomials  (Ortho-normal) 

This  case  is  the  orthogonal  conditions  for  Case  2 and  by  Eq.  (249)  sec  (5) 


G(M)  - MB  (M) 

(N+l)x3  ® 

where  by  Eq.  (250)  sec  ( ) 

1 0 

B (M)  = 0 1 

g 

0 0 


(77) 


-N(N+2) 

12 

0 

1 


(78) 


or  by  Eq.  (77) 

M = G(M)B~^(M) 
8 


(79) 


Using  Eq.  (79)  in  the  transpose  of  the  fitting  function  F matrix  of  Eq.  (40) 


F = T = DT^(B)b"  (M)G  (K) 


By  Eq.  (42)  the  inverse  relation  becomes 

T -1  -1  T -1  -1 

(T  T)  = (M  M) 

and  by  Eq.  (79)  and  its  transpose 
M^M  = B~^(M)G^(M)G(M)b“^(M) 


(80) 


(81) 


(82) 


and  the  inverse  is 

(M^M)"^  = B (M)(g^(M)G(M))“^  B^(M) 


(83) 


By  Eq.  (77) 

T — T T 

M = Bg  (M)  G (M) 

3x3  3x(N+l) 

By  Eq.  (44) 

“>■ 

The  psuedo  inverse  of  M by  Eq.  (83)  and  (84)  is 

(M^M)“V  - B (M)[g’^(M)G(M)]"^  g'^(M) 

8 


(84) 


(85) 


(86) 
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(87) 


Using  Eq.  (86)  In  Eq.  (85) 

b'^(M)t'J^  a>-  [g'^(M)G(M)]"^  g’^(M)z> 

The  matrix  G^(M)  by  Eq.  (77)  is 

T T T 

G (M)  = BgM 

and  by  Eq.  (78)  and  Eq.  (41)  in  Eq.  (88) 


1 

0 

0 

<1 

1 4 ' 

g’’(M)  = 

0 

1 

0 

0 

-N(N+2) 

12 

0 1 

0 

where  Lc  is  the  linear  convolution  matrix 


Eq.  (89)  becomes 


g'^(m) 

3x(IH-l) 


(88) 


(89) 


(90) 


(91) 


The  parameter  estimate  in  the  new  base  now  becomes  by  Eq.  (87)  and  Eq.  (255) 
sec  (5)  for  the  inverse  metric  matrix 


1 

N+1 


as 


^ =j  0 
0 


0 

12 


N(N+1) (N+2) 
0 


0 

180 


N(N+1) (N+2) (N  +2N-3) 


■(M)z^ 


(92) 


The  midpoint  variance  matrix  of  Eq.  (46)  is 

= A T"^  (m'^M)"^  T"’’ 

aa  hou  hou 

and  by  Eq.  (83)  in  Eq.  (93) 

= A T"^  B (M)  G^(M)G(M)"^  b'^(M) 

aa  hou  g \ \ / g'  ' hoi 


(93) 


(94) 
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(95) 


Do  the  base  change  on  Eq.  (94)  and  obtain 


T * 

aa  hou  aa  hou 


E-i? 


where 


Yl~~  ‘ B (M) 
aa  g' 


1 

N+1 

0 

0 


0 

12 


N(N+l)(N+2) 

0 


0 

180 


N (N+1) (N+2) (N  +2N-3i 


Bg(M)  (96) 


The  a^  of  Eq.  (92)  is  for  the  orthogonal  midpoint  polynomials.  If  we  use  the 
ortho-normal  relations  given  by  Eq.  (258)  sec  (5)  as 


S = M B (N) 
s 

(N+l)x3  (N+l)x3 
where  by  Eq.  (260)  sec  (5) 

f 1 

I /N+1 


B (N)  =j 

= I 


By  Eq.  (97) 

SB~^  = M 
s 


0 

0 


0 

/12 


/N (N+1) (N+2) 
0 


-N  (N+2)/IW 


12  N(N+l)(N+2)(N^+2N-3) 


0 


12[n(N+1) (N+2) (N^+2N-3)] 


(97) 


(98) 


(99) 


and  Che  metric  is 

T -1  T -1  -T  -1 

M M = B S SB  = B B 
s s . s s 


(100) 


and  the  Inverse  of  Eq.  (100)  is 

T -1  T 

(M  M)  = B B 
s s 


(101) 


and  since  these  are  ortho-normal  polynomials  the  metric  is  the  Identity, 
that  is 

s'^S  = I (102) 

3x3 


hence 


(103) 


T -1 
(MM) 


By  Eq.  (44) 


T 

B B 
s s 


<l>  ■ 


The  psuedo- Inverse  of  M is  by  Eq.  (97)  and  Eq.  (103) 

T —1  T T — T T T 

M*  = (M^M)  M » B B B B S - B S 
s s s s s 

Using  Eq.  (105)  in  Eq.  (104) 

• «>  - 


or 


B ^ T,  a, 
s hou 


= a®^  = S 


the  parameter  in  the  new  base  a®^  is  given  by  Eq.  (107)  or 

M « SB'^ 
s 

where  by  Eq.  (97),  Eq.  (98)  and  Eq.  (41) 


1 

0 

0 


<M)  1 

WfT 

b22<^)c 


bi3<^)  1 - 


(104) 


(105) 


(106) 


(107) 


(108) 


(109) 


bi3<«)l  + b33<c 


where  B by  Eq.  (98)  is 
8 


'11 


22 


(110) 


i 

t 

I 

II 

l:ti 


J 


I 
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Eq.  (106)  to  multiply  2^  to  obtain  a>  . 
in  Eq.  (93)  yields 

(111) 

Case  6.  Front  to  Back  or  (Backward)  Orthogonal  Polynomials 

This  case  will  not  be  presented  here  since  it  is  similar  to  the  forward  case 
except  for  the  alternating  sign  changes  mentioned  before.  The  congruent 
transformation  on  the  (rTp)-!  matrix  is  given  by  Eq.  (278)  sec  (5)  to  obtain 
the  diagonal  metric  matrix. 

Case  7.  Backward  Exponential  Weighted  Polynomials 

This  case  will  consider  the  fitting  of  the  data  by  exponentially  weighted 
polynomials  with  the  indices  running  from  the  front  of  the  data  (real-time 
or  front-time)  point  to  the  back.  The  measurement  data  will  be  assumed  to 
go  back  in  time  to  negative  infinity,  with  zero  values  prior  to  the  actual 
time  the  data  is  available.  By  Eq.  (169)  sec  (5)  the  time  variable  will  be 
indexed  as 

t(i,  y)  = ho(6  + i)  - hoy  (112) 

with  the  span-generating  index  i taken  to  be  zero  in  this  case  (for  the 
recursive  case  it  will  not  be  zero).  By  Eq.  (171)  sec  (5)  the  fitting 
functions  are 

= (1,  t,  t^  ...  ) e (113) 


Equator  (109)  is  to  be  used  with 

The  variance  matrix  by  Eq.  (101) 

» AT“^  B B^  t”^  . 

aa  hou  s s hou 


or 

The  index  y (for  il  = 0)  by  Eq.  (176)  sec  (5)  is 
qy/2^  T^(B)D(ho) 

where  by  Eq.  (177)  sec  ( ) 

= (1,  -Y,  Y^,  -Y^,  Y^  ...  ) 

and  packagewise  by  Eq.  (178)  sec  (5) 

Tg  - F - e rT^(6)D(ho)  (117) 

(N+l)xd 

and  the  discrete  metric  by  Eq.  (179)  sec  (5)  is  for  the  3x3  case 


(115) 


(116) 
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(118) 


T T 

e e 

3x3 


D(ho)  T^(B)  r"wrT^^(6)D(ho)  e 


The  inverse  is 

T -1 
(T  T ) ^ • 
e e 


—1  T -1  -T 

T.  (r  wr)  ^ T.  e 

hou  hou 


-ao 


(119) 


The  weighted  metric  inverse  is  given  by  Eq.  (343)  sec  (5)  as 


T -1 

(r  wr) 


(1-0) (i+e+0^)  |(i+0)(i-0)^ 


|(  1+0) (1-0)^ 


(1-9)- 


(1-0)^(1+100+90^ 

40^ 

(l-0)^(l+30) 

40^ 


(1-9)^ 

2 

(l-0)^(l+30) 

40^ 

(1-9)^ 

49^ 


(120) 


where  0 is  given  by  Eq.  (305)  sec  (5) 

0 = e 

and  a is  related  to  the  time  constant  in  the  exponential  of  Eq.  (113). 

The  estimate  of  the  parameters  is  given  by  Eq.  (31)  as 
T„  .-1  „T 


(3>=  (T^T^)  T^  z (N+^  = T*z^ 

3x3  (3xN+l)  (N+l)xl 


(121) 


(122) 


Notice  that 


T z 
e 


(^  = 


3x" 
where 

p » 00  - N+1 

and 

z ('^  = 


3x(N+l)  3x/~| 


z (I«-l>\ 
0 (/">/ 


(N+^\ 

(Oj 


(123) 


(124) 


The  psuedo-inverse  of  T^  is 

T*  » (T^T  )"^  T^ 
e e e e 


(125) 


and  by  Eq.  (119)  and  Eq.  (117) 
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T*  » (r^wr)"^  6 

3xN+l 


(126) 


Using  Eq.  (126)  in  Eq.  (122) 


\ou^>  “ = (r’^wr)"^  z>  e 


-ao/2 


(127) 


where  by  Eq.  (174)  sec  (5) 

In,  aho3  ato 

0 ^ g — j-  ^ g — 


(128) 


and  if  t =0 
o 

ao 

0 2 =1 


The  vector  is  by  Eq.  (180)  sec  (5) 


3x(N+l) 


-N  -(N-1)  ...  -2  -1 

(N-1)^  ...  2^  1 


and  the  W ' matrix  by  Eq.  (151)  sec  (5)  is 


Note  also  that  Eq.  (125)  by  Eq.  (181)  sec  (5)  is 


<^l)n 


where 


= - <N+ 

3x(N+l)  r 

<^l)n 


f+l)nL 


<N+1)  n - (0,  1,  2,  3 ...  N) 
and  is  the  linear  convolution  matrix. 


(129) 


(130) 


(131) 


(132) 


(133) 
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rV/2 


y> 


Lc  z (N+1 


(134; 


Using  Eq.  (128)  and  Eq.  (126)  in  the  vector 
<^l)  n^  z (n+^ 

-<^l) 

<f5+l)  n^ 

L 

Thus  is  obtained  by  the  products  of  Eq.  (120)  and  Eq.  (128)  and  Eq.  (123). 
The  variance  is  given  by  Eq.  (7)  for  scalar-matrix  noise  as 

(135; 


Lc  W^'  ‘ z (n+^ 


Z--  = T*  T*^  o 
aa  e e V 


Using  Eq.  (126)  and  its  transpose  in  Eq.  (129) 
^aa 


T -1  -1 

(r  wr)  T. 

hou  hou 


--ao 

e o. 


(136) 
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Section  12  UNCONSTRAINED  UNWEIGHTED  STATE  ESTIMATION 


This  section  relates  the  state  estimation  relations  to  the  parameter  estimation 
relations  of  the  previous  sections. 


By  Eq.  (13)  sec  (IQ) 

z(N+iN  = F a(d>  + v(N+l>  = F a(d>  + 
(N+l)xd 

By  Eq.  (13)  sec  (4) 


i(<f> 


(1) 


x(t) 

X 

s 

<:J(t) 

<f(t)V^ 

^(d-pl) 

<f(t)Vj'^"^ 

(2) 


For  the  class  of  fitting  functions  with 
</(o)  = (1,  0,  0 ...  ) = ^ 

We  have  by  Eq.  (2) 

X(^g  = F^(o)a(^ 
dxd 

Solving  for  the  parameters 
a(^  = F"^(o)  X(o)  (^ 

Polynomials  Back  to  Front 

The  discrete  state  transition  is  given  by  Eq.  (160)  sec  (4) 
where  , 


(3) 


W 


(5) 


t = t. 


with 


or 


t « t,  + Nho 
b 


t = Nho 

hence  Eq.  (160)  sec  (4)  becomes  for  the  3x3  case  of  polynomial  functions 

1 Nho 


♦ (N,o)  = $ 

X X 


N 


2 2 
N ho 


1 

0 


2 

Nho 

1 


(6) 
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and  the  inverse  by  Eq.  (164)  sec  (4)  is 

fl  -Nho 


The  discrete  state  transition  matrix  of  Eq.  (6)  by  Eq.  (158)  sec  (4)  yields 
X(N)  X(o)  (3>^  ( 

«(o)  X(N)  ( 

Using  Eq.  (9)  in  Eq.  (5) 

a(^  = F"^(o)  x(N)  ( 

The  measurements  of  Eq.  (1)  are 
rz(o)l  rx(o)!  [v(o)l 


z(o) 

x(o)| 

2(1) 

x(l)j 

• 

» 

1 . 1 

z(N) 

* 

'x(N) 

L j L 

hence  by  Eq.  (10) 


lv(N) * 

L J 


X (N+:^ 


= F ^ = F F"^(o)  x(N) 

(N+l)x3  (N+l)x3  3x3 


which  relates  the  states  of  the  function  at  time  corresponsing  to  N to  the  function 
values  at  the  (N+1)  time  points.  Inverting  Eq.  (12) 

x(N)  F^(o)  F*  X (N+^  (13) 

3x(IH-l) 

The  above  relations  will  be  investigated  for  the  monomial  polynomial  base  and 
the  exponentially  weighted  polynomial  base. 

Consider  first  the  3x3  matrix  of  Eq.  (10)  by  Eq.  (7)  and  Eq.  (1$  sec  (4) 
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F ~^(o) 

V X 


1 
0 
L 0 


0 

1 


0 

0 


0 1/2 


‘1 

-Nho 

N^h^ 

0 

1 

-Nho 

.0 

0 

1 

(14) 


or 


f"^(o)  4> 

V X 


0 

0 


-Nho 

1 

0 


Also  the  inverse  of  Eq.  (15)  is 

1 Nho 


<J>^  F (o) 

X V 


1 

0 


n\^/2 

o 

-Nho 

1/2 


N^h^/2 

o 

Nho 

1 


By  Eq.  (30)  sec  ( ) for  a forward  cycling  filter 
T*  = D"^(ho)  t"^(B)  N* 


(15) 


(16) 


(17) 


For  the  special  case  of  tp=0,  the  matrix  product  of  Eq.  (13)  becomes 

N N -1  T -1  T 

$ F (o)F*  = $ T (o)  D (ho)(N  N)  N 

XV  XV 


(18) 


The  intermediate  matrix  product  term  of  Eq.  (18)  is 


D(ho)  t“^(o)  <t  ^ 

V X 


-Nho 

ho 


2 2 
N'^ho 

-Nho 

ho 


(19) 


The  matrix  product  term  of  Eq.  (12)  is  for  the  forward  cycling  filter  by 
Eq.  (14)  sec  (11) 

F F~^(o)  4>"^  = ND(ho)  F~^(o) 


(20) 


By  Eq.  (16)  sec  (D)  and  Eq.  (19) 
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1 (N-1)  (N-1)^  j 

,,2 


-Nho  N^ho^/2 

-(N-l)ho  (N-l)^ho^/2 

-(N-2)ho  (N-2)\o^/2 


ho^/2 


Note  that  Eq.  (21)  used  in  Eq.  (12)  relates  the  states  at  Nho  time  points  ahead 
to  the  function  values.  The  normal  Taylor  series  expansion  is  to  relate  the 
function  values  to  the  initial  states,  that  is  for  example 

, 2 


x(l)  = x(o)  + x(o)ho  + x(o)— ^ 


x(2)  = x(o)  + x(o)zho  + x(o) 


(zho^) 


x(N)  = x(o)  + x(o)Nho  + x(o) 


or  packaging 

~x(o) 
x(l) 
x(2)  = 


h^o/2 


zho  (zho)  /2: 


1 Nho  (Nho)  /2; 


The  above  matrix  can  be  compared  with  the  matrix  of  Eq.  (21). 
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Observe  that  the  state  transition  matrix  for  the  polynomials  yields 
x(l)  = $ x(o^^  = 

x(2)  = x(.2^^  » ^ x(o^^ 


x(N)  = x(N^s  = 


or 


x(o) 

x(l) 

'I' 

x(2) 

■ 

x(oJ>^ 

x(N) 

By  Eq.  (11)  and  Eq . (1) 


X (N+^  = F a (|> 
(N+l)x3 


and  by  Eq.  (5)  in  Eq.  (25) 

X (N+l>  = F F‘^(o)  x(o)  (|>^ 

By  Eq.  (24)  and  Eq.  (26) 


F F~^(o)  = ! 

V , 


(N+l)x3 


we  have  also 


and  using  Eq.  (29)  In  Eq.  (24) 


(N+^« 


.-N 


.-N+1 


"4 


-N+2 


4>  ^ 


am  (i>. 


which  is  the  same  as  Eq.  (12);  hence,  by  Eq.  (12)  and  Eq.  (30) 


-1  -N 

F F (o)  4> 

V X 


(N+l)x3 


-N+1 


^4 


"<1 


Using  Eq.  (12)  in  Eq.  (11) 

z (N+^=  FF~^(o)  x(N)  (|>g 


+ V 


(N+^ 


= FF~^(o)  x(N,N)  + z(N^ 


-N  - 


where 


x(N,N)  = $^F^(o)F*  z (N-^ 


where  x(N,N)  (|>  is  the  unweighted  estimate  of  the  states  at  t-Nho  based 
on  using  all  of  the  first  N+1  measurements,  or  data  up  to  time  Nho. 


The  matrix  product  terms  of  Eq.  (34)  are  given  by  Eq.  (18)  and  by  Eq.  (IJ) 


$^F^(o)F*  = $\^(o)D“^(ho)(N^N)"^  N*^. 
By  Eq.  (10)  we  have 

i(N,N)  (^g  - -I>”f^(o)  a (p 


and 


i(N,N)  = 4)  a fJ-(o)($^)''^ 


and  the  variance  matrix  under  a base  change  transforms  via  a congruent 
transformation,  that  Is 

E(i(N,N)^<^i(N,N)) 

V ^aa  V ' ' 

By  Eq.  ( ) sec  ( ) In  Eq.  (39) 

‘>\(o)D'^(ho)(N^N)“V^(ho)Fj(o)($**)^a^ 


Section  13  RECURSIVE  PSUEDO  INVERSE  AND  PROJECTORS 

Given  a matrix  T of  rank  d < k,  the  psuedo  inverse  of  T is  given  as 

* T -1  T 
T = (T  T)  T 

dxk  dxd  dxk 


(1) 


thus  the  computation  of  the  psuedo  inverse  can  be  obtained  as  the  inverse  of  a 
symmetric  (Grammian)  matrix  T T.  If  we  now  have  a new  matrix  (k  + l)xd  in  size, 
with  T as  a submatrix 
kxd 


T 

(k+l)xd 


T 

kxd 


t 
k+1 


(2) 


the  psuedo  inverse  is  given  as 

* T -1  T 

T = T) 

dx(k+l)  dxd  dx(k+l) 


(3) 


where 


T 

dx(k+l) 


and  the  symetric  Grammian  is 


cr,T 

T , 

t 

dxk 

k+f 

T 

T T 


T 

T T + t 


k« 
^\t 
‘k+1 


(A) 


(5) 


d(k+l)d  d(k)d 

the  previous  Grammian  plus  a dyad. 

The  inverse  of  the  new  Grammian  as  a function  of  the  inverse  of  the  old  Grammian 
is  given  by  Householder  as 


(tTt)-i 

d(k+l)d 


_ (.rT.j.^-1 

d(k)d 


[I  - 


k+1 

X 

X+1^ 


(T^T)"^ 

d(k)d 
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(6) 


k+1  d(k)d  ^ k+1 

If  we  partition  the  new  psuedo  inverse  of  Eq.  (3)  as 

* 

T 

= T*(k  + 1),  t*^  1 

(7) 

dx(k+l) 

|dxk  ^ k+lj 

= r T^;  (T^T)"^  b 1 

d(k+l)d  dxk  d(k+l)d  k+1  | 

(8) 

Using  Eq.  (6)  in  Eq.  (8) 

,v  r * 

1 / m / 


T 

dx(k+l ) 


(T"  - (T^T)'^t^<^ 
dxk  d(k)d  ^k+1 


T* 


1+  <t(rT)“^t/ 

k+l  d(k)d  k+1 


(T^T)'^t 
d(k)d  yk+l 

1 + 

k+l  d(k)d  ' k+l 


Note  that 


T"(k  + 1)  = T - (T  T)"  ys^  T 

dxk  dxk  '^k+1^  dxk 


T* 

■^k+N 

Xt  (T^T)‘^t 


y 

/ k+l 


t"  ^ = (T^T)“^t^  (11) 

1 + <t(T^T)"^^ 
k+l  ''  k+l 

If  we  consider  T as  d column  vectors  in  k space,  then  the  k x k matrix  projector 
kxd 

onto  the  subspace  spanned  by  the  d vectors  is 


The  Orthogonal  compliment  projector  is 
P * = I - P * 

tt”  tt" 

kxk  kxk 

The  expanded  matrix  is 


T 

(k+l )xd 


• T 
I kxd 
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and  the  expanded  projector  by  Eq.  (7)  is 

* * V 

T (k+1),  t (d> 


TT 

(k+l)(k+l) 


TT  (k+1) 
kxk 


k+1 


-^t  T*(k+1) 


k+1  dxk 


k+1 


k+1 


The  kxk  submatrix  of  Eq.  (15)  is 


T T*(k+1)  = TfT"  - (T^T)' 


kxk 


kxd  dxk  d(k)d 


dxk 


or 


TT 

k(d)k 


1 +<^(T^T)“^^ 

* *TN.  * 

(k+1)  = TT  - T ^ <t  T 


kxk  kxd  k+1 


1 +<Jt  (T^T)‘^N 

1 


4 


k+1 


k+1 


The  other  submatrices  are  by  Eq.  (11) 
= T(T^T)"^^ 


Tt  = T(T  T)  t> 
k+1  ^ k+1 

1 +<t(T^T)'^^ 
k+i  'k+l 


The  inner-product  of  Eq.  (15)  is 


<tt 


k+1  k+1 


= *^'*^^<;t(T^T)'^^' 


k+1 


1 + <Ct(T^T)"^^ 

k+1 


k+1 


(15) 


(16) 


(17) 


(18) 


1 


■ 1 

4 -S 


•i  i 


■'  j 

-V  j 


j 


i; 


and  the  remaining  term  of  Eq.  (15)  is 


k * 


W 1 


a. 


L 


A = 1 + 

Note  that  the  commute  of  Eq.  (20)  is 


^t(T^T)'^^ 


(21) 


•k 

T T 


(dxk+1) (k+l)xd 


I = 
dxd 


T*(k+1),  t*^  ] f T 

dxk  k+1  j kxd 

L<^. 


(22) 


or 


I = T (k+1)  T + 

kxd  k+ 1 


* 

r ( 

dxk 


(23) 


(B)  Recursive  Psuedo  Inverses  of  Full  Rank  Expanding  Rectangular  Matrices  Times 


Full  Rank  Square  Weighting  Matrix  and  Recursions  on  Their  Projectors. 


The  matrix  relations  in  this  section  are  useful  in  weighted  least  squares. 
Consider  the  product  of  a k x k full  rank  matrix  M times  the  full  rank  matrix 
T , or 


M T 
kxk  kxd 


= T, 


M 

kxd 


(24) 


and 


T T T 


(25) 


The  generalized  inverse  is 
T, 


M 

dxh 


T - 1 T 


dxd 


M 

dxk 


T T -ITT 
(T  M Ml)  T M 


(26) 

(27) 


The  kxk  matrix  orthogonal  projector  onto  the  space  spanned  by  the  columns  of 
Eq.  (24)  is 


* T T -ITT 

T„T„  = MT  (T  M MT)  T M 
M M 


(28) 


kxk 


The  expanded  T matrix  times  the  corresponding  (k+  1)  x (k+  1)  M matrix  is 


‘M 

(k+1 )xd 


M 


(k+l)(k+l) 


T 

(k+l)xd 


(29) 


with  psuedo  inverse 


‘M 

dx(k+l) 


(T 

dx(k+l) 


MT) 


-1 


(30) 


(k+1)  (k+l)xd  dx(k+l) 


i 
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The  expanded  T matrix  and  its  transpose  is  given  in  partitioned  form  in  Eq.  (2) 
and  Eq.  (4),  hence  it  is  convenient  to  partition  M as 


M 

(k+l)(k+l) 


M(k+1) 

kxk 

4 

k^l 


and  the  expanded  Granunian  is 


M^(k+1) 


T 

MM 

(k+l)(k+l) 


kxk 


k+1 


(k+l)k+l 


k+1 


|tl(k+l) 


k+1 


m 


(k+1) 


Mj(k+l)M(k+l) 

^(k+1)  + m ^ ^ 


r 

k+l^ 

rp  U 


k+1 


M^(k+l)m 


k+1,  k+1 


m 


(k+1) 


+ m 


or 


G(k+1) 

kxk 

gvk> 

V k+1 

^g 

g 

k+1 

k+1,  k+1 

(k+l)(k+l) 


Using  (33)  in  square  matrix  inverse  term  of  Eq.  (30) 

.-1 


T 

(T  G 


T) 


dxk+l(k+l)  (k+l)xd 

" r 


'T  T \ 

=!  1 T , t) 

■ dxk  / 


, L 


G(k+1) 
j kxk 


! 


T"G(k+l)  T + y <^g  T + T^g, 
d(k)d  k+1  k+d 


\ -1 


T + ^ g I 

lr+1  lr+/>  ^ I 


The  sums  of  the  three  dyads  of  Eq.  (34)  can  be  written  as 

g 

T 


aujiid  uj.  uiic  uiiLcc  uydua  ui 

^<gT  + T^^<t  + t^4, 

, T^g>  , t ^ 

Xdxk  X kxl  ^ 


<gT 


= BC 
d(3)d 


(31) 


(32) 


(33) 


(34) 


(35) 
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T 


where 


t)| 

c = 1^)8  1 

dx3  ' ' k+1,  k+lj 


Equation  (34)  can  be  written  as 
T -1 

(T  G T) 

d(k+l)  (k+l)(k+l)  (k+l)xd 

= [T^  G(k+1)T  + B C^]"^ 

dxk  kxk  kxd  dx3  3xd 

= (T^G(k+l)T)'^  X 


•l  - B (I  + C*  (T^G(k+l)T)"^  B]"^  C^(T^G(k+l)T)‘ 

'^d  dx3  3x3  3xd  dxd  dx3  '' 

Equation  (38)  comes  from  Householders  inversion  of  modified  matrices  (ref. 39) 
which  is 

(A  + UV^)  = a'^  - a‘^U  (I  + V^A"^U)"^V^A"^  (39) 

If  we  impose  the  constraint  (inspired  by  knowledge  of  known  properties  when 
applied  to  variance  matrices)  that  the  kxk  submatrix  cf  the  expanded  Grammian 
be  equal  to  the  previous  Grammian,  by  Eq.  (32) 


G(k+1)  = G = M^(k+1)  M(k+1)  + p 

-.1-  1 i_  I l_  1 1.  ' 


kxk  kxk  kxk  kxk 

then  we  can  obtain  the  expanded  psuedo  inverse  as  a function  of  the  previous 
one.  Let  ^ _ 

dxk+l  dx(k)  k+1  (^ 


" T -1  T T ~ 

_ (t^GT)  T M 

d(k+l)d  (k+1)  X (k+1) 


where 


T T T 

T M = (T 


= (t"^,  ^ M'^(k+1)  m) 

^m  m 

= (TV(k+l)  + t(d)><(m,  + t)>m^ 

dxk  dxk  dxl  / 


The  first  submatrix  of  Eq.  (41)  by  Eq.  (42)  and  Eq.  (34)  is 

T^*(k  + 1)  = (T^GT  + (tV  + 

dxk  —V 

T*(k+1)  = (T^GT)"^^  - B[I  + C^(T^GT)"^B1‘^c'^(T^GT)"^tV  + <ni) 


(43) 


_ ,p*  ^ (T^GT)"^  B[I  + C^(T^GT)"^B]"^cV 

ID  in 

dxh 

+ (T^GT)’^il  - B[I  + C^(T^GT)“^B]~^C^(t'^GT) 
The  (k  + l)th  column  vector  of  Eq.  (41)  is 

C ^ = (t'^’gT)’^  [t’’m^\  + t(^ml 

^+1  d(k+l)d  dxk 


-1^  t)<m 

J 


(44) 

(45) 


= (T^GT)'^I  - B[I  + C^(T^GT)"^B]“^C^(T^GT)"^  [T^M^  + tNm] 
d(k)d  ^ ^ / J 


(46) 


Equation  (45)  and  (46)  are  quite  messy.  They  can  be  simplified  by  assuming  G 
is  diagonal  which  implies  that  M is  an  orthogonal  matrix.  They  can  be  further 
simplified  by  assuming  that  M is  diagonal  which  is  equivalent  in  the  variance 
applications  to  the  uncorrelated  assumptions. 


dUllipUXUlia  IiCllL.C;XUL  Ull  WXXX  Ul 

^ = m{^  = M^ 


-1 


The  assumptions  henceforth  will  be  assumed, 

g 

and  the  matrix  of  Eq.  (38)  becomes 

(T^GT)"^  = [T^GT  + t/  ^ g ] 

d(k+l)d  d(k)d  ' ^ k+1,  k+1 

and  by  the  Householder  inversion  of  Eq.  (6) 

(tTgt)-I  _ (t'^gT)"^  [I  - tS  /t(T^GT)'^g] 
d(k+l)d  


(47) 


1 + g<^t(T’^GT)“^^ 


The  transpose  of  Eq,  (42)  becomes 

m 


tV  = IT^ 


dxk+1  dxk 


k+1.  1+1 


(48) 


(49) 


The  expanded  space  psuedo  inverse  of  Eq.  (41)  by  Eq.  (48)  and  (49)  is 

T - 1 v y "ff 
(t‘cT)  ‘t^  <t  s 


T*(k*l)  = 4 - 


dxk 


dxk 


1 + g^t(T'^GT)'^^ 


(50) 
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and  the  (k  + l)st  column  of  Eq.  (41)  is 
= (T'^GT)'^t^  g 

^+1  V] 

1 + g \t{T^GT)’^t) 


where 


The  expanded  space  (k  + 1)  x (k  + 1)  matrix  projector  for  the  diagonal  constraints 
above  and  using  the  transpose  of  Eq.  (42) 


I MT 
I kxd 


M 

(k+1 )xd 


T„  T*  = r MT  ] 

(k+l)xd  dxk+1  I kxd  i 


m < t 


Tfi(k+1), 


MTT"(k+l) 
kxd  dxk 


m^"^t  T”(k+]) 


!\  I 


m <.  t t 


k+1 


The  first  submatrix  of  Eq.  (54)  is  by  (50) 
MTT“(k+l)  = MTtJJ  - MT(T^GT)‘^t  t T*g 

1 + g t(T'^GT)'^^ 

= Vl  ■ V 

‘<'<>1'  Z ITT 

1 + g < t(T"GT)  "t)> 

The  first  row  second  column  vector  of  Eq.  (54)  is 
MTt* '>=  MT[(T'^GT)“^t>  Vg 

1 + 84  t(T'’^GT)~^t  > 


= > Vg 


1 + g <t(f%T)’^t> 
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and  due  to  the  symmetry  of  the  projector  of  (54) 

m<^T*(k+l)=  t t’' 

Ag  dxk 

The  second  row,  second  column  element  of  Eq.  (54)  is 
m <tt”')  = g t(T'^GT)~^t) 

1 + g ^t(T^GT)"^t^ 


= Ag  - 1 

Ag 


where 


= 1 + g < t(T''’GT)"^t  > 


Thus  the  expanded  space  projector  in  terms  of  the  previous  projector  is 


(k+l)(k+l) 


* *T  ^ * 


Ct  > * 


! / * 

k+1  Ag 


Ag  - 1 
Ag 


II  Recursive  Psuedo  Inverse  of  Fixed  Size  Rectangular  Matrices  Generated  by 
Shifting  all  Rows  Up,  the  First  Row  Out  and  Adding  a New  Last  Row,  and  Recursion 
on  Corresponding  Projectors. 

This  section  considers  a two  index  sequence  of  vectors  in  d space 

t (i+n)  (61) 

where  i is  an  open  ended  running  index  and  n runs  from  0 to  N,  then  i goes  to 
i+1,  etc.  The  open  ended  string  of  vectors  are 

/^t(i+o) 

Wt(i+l)*v 

N+l'S  • /n+1 

Vectors/  • ^ Vectors 

Kt(i+N)  \ 

^ t(i+N+l)/ 
t(i+N+2) 


4t(i+2N) 


pp  1 1 TWH  illTITWIiirjiBI 


We  want  to  psuedo  invert  the  (N  + 1)  x d matrices  (full  rank  assumptions  for 
simplicity) . 

T(i)  =^<t(i+o)  ” 

(N+l)xd  • (6: 

<t(i+N) 

L _■ 

and  the  succeeding  fixed  size  rectangular  matrix  of  the  same  size  and  rank  d, 
defined  as 

'<'t(i+l) 

i 

T(i+1)  = • (6; 

(N+l)xd 

<t(i+N) 
j<  t(i+N+l) 

Partition  T(i)  as 


T(i)  = ^t(i+o) 


T(i+1)  = 


/ \ 

f Nxd  I 

^ ^t(i+N+l)  I 


where  the  vectors  in  the  intersection  of  the  two  sequences  is  designated  Tn- 
The  psuedo  inverse  of  Eq.  (62)  is 

T”(i)  = IT'^(i)  T(i)l"^T'^  (( 


dxN+1 


d(N+l)d  dxN+1 


The  rank  d assumption  implies  that 
N + 1 2 d 


The  inverse  of  the  symmetric  matrix  is 


\_T'^(i)  T(i)']"^  = [t)  ^ ° (t  + 

= (T^T  )’^  I - t)o°d(T^T  )’^ 

n n f \ n n 


where 


/t  =<t(i+o) 
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The  psuedo  inverse  of  (66)  is 

T*(i)  = lT'^(i)T(i)]'Mt)^,  tJ] 

dxn 

= (t*(ij)  T*(i)) 

' ° dxn 

The  first  column  vector  of  Eq.  (70)  by  Eq.  (68)  is 

/(!)>„=  (tXr't), 

1 . Ct  „ 

^ o o 

The  second  matrix  of  Eq.  (70)  is 

T*(i)  = \ °<^  < 

dxN  dxN  ^ T 1 V 

1 +<t  (T^T  )"\> 
d\  n n /o 


where 


TIT* 

t"  = (T  T )"  t" 
. n n n n 

dxn 


The  generalized  inverse  by  Eq.  (71)  and  Eq.  (72)  in  Eq.  (70)  yields 

* Ft  -1v  * T 

T (i)  = \ij2n^o  ’ ^>o<^^n' 


dxN+1  L 


where 


The  projector  onto  the  space  of  T(i)  is 


T(i)  T''(i)  =1: 

(N+1)(N+1)  j 


= !Vd)t  i |'(t'^T  )'^tS  , T*  - (T^T  )"^^^t  t" 

I V / ! _n_n_  /o  II  ^ n n n 

i ^ A A 


c A -1 
o 


*T  V * *T  °/ 

T t>  T T - T tV  <t  T 

n ' o n n n / n 


The  psuedo  inverse  of  T(i  + 1)  is 

T*(i+1)  = [T^(i+1)  T(i+1)]'^  T^(i+1) 


dx(N+l) 


dxN+1 
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1 


The  Grammian  is 

T^(i+1)  T(i+1)  = T^T 
n n 


N+l 


(78) 


where 


N+r 


(i+N+1) 

By  the  Householder  inversion  relation 


(T 


N-t-1 

‘(i+1)  TCi+l]"^  = (T^T^)"^  (I  - t^ 


(79) 


(80) 


M+l 


where 


N+l  ] 


N+l 


and 


T^(i+1)  = [T 


^ N«1 


a+ij 

T*(i+1)  = [T''(i  + 1); 

dx(N+l)  dxN 


A 

; t ; J 

'N+l 


(81) 

(82) 

(83) 


Using  Eq.  (80)  and  Eq.  (82)  in  Eq.  (77)  the  first  matrix  of  Eq.  (83)  is 


* * T - 1 \ * 

T (i+1)  = T - (T  T ) t>  T 

J nr  I’  'in  <t . i''  n 

dxN  N+l 


(84) 


AN+1 

the  (N  + l)st  vector  of  Eq.  (83)  is 

T - 1 \ 

F T ) 
n n r 


t = (T^ 

/N+l  '•  - 


N+l 


(85) 


AN+1 


The  generalized  inverse  by  (84)  and  (85)  in  (83)  yields 

* * T - 1 \ * T - 1 \ 

T (i+1)  = [T  - (T  T ) t)  \t  T , (T  T ) t>  ] 
n n n ^+1  ' " ” " ^N+l 

M+1  

and  the  projector  onto  the  space  of  T(i  + 1)  is 


(86) 
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T(i+1)  T*(i+1)  =/t\  T*(i+1),  t/ 

(N+1)  (N+1)  L ^ ' 

rv 

T t"  - T"^t^  <t  t"  T 
n n n / \ n n / 


NxN 


QJ±1 

M+l 


/ t T* 
Vl-T§ 


AN+1 


N+1 


m. 


M+l 

M+l  - 1 
M+l 


(87) 


Equation  (86)  and  (87)  obtained  the  psuedo  inverse  of  the  updated  matrix  and  the 
projector  as  a function  of  the  corresponding  matrices  in  the  intersection.  An 
Alternate  derivation  could  be  obtained  using  the  shift  up  and  out  matrix  of 
Eq.  (5)  Sec.  (6)  or 


S T(i) 
uo 

(N+l)(N+l)xd 


and 


1 

<'t(i+l)  ' = 

• ! 

Xtd+N) 

. 

^ o ! 

\ i 

j 

(88) 


T(i+1) 

(N+l)xd 


S T(i) 
uo 

(N+l)xd 


(89) 


An  extension  of  Householder's  inversion  lemma  to  the  psuedo  inverse  of  rectangular 
matrices  plus  a dyad  would  yield  a direct  result,  however  this  route  will  not 
be  pursued  at  this  point.  The  psuedo  inverse  of  Eq.  (89)  is 


T''(i+1) 

dx(N+l) 


[T^(i) 


T \ / -1  T T 

S S T(i)+A  t]  T (i)  S 
uo  uo  / \ u 

dxd  N+1  d(N+l) 


where 


ST  S 
uo  uo 


(N+1) (N+1) 


A less  powerful  approach  can  be  used;  let 


T(i+1) 

(N+l)xd 


T(i)  - 
(N+l)xd 


(90) 


(91) 


(92) 


or 
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(93) 


L:. 


T(i+1)  = T(i)  + l- 


(N+l)xd 


+1 


N?1 


T(i+1)  = T(i)  + E F 

tN+l)xd  (N+l)xd  (N+l)x2xd 


L<^' 


(94) 


where 


<^=  (1 


, 0,  0 . . .) 


T (i+1)  = T (i)  - T (i)  E(I  + FT  E)  FT  (i) 
dx(N+l)  dx'N+l) 


Equation  (95)  is  the  rectangular  analog  of  Eq.  (39). 


T (i+1)  = T (i)  - T (i)  E (I  + FT  (i)E)  F T (i) 
dx(N+l)  dx(N+l)  2x2  2xd 


(95) 


T(i+1)  T"(i+1)  = (T(i)  + EF)  T"(i)  ft  - E[I  + FT*E]*FT*  \ 
(N+1)(N+1) 


} 


= T(i)  T*(i)  |l  - E[I  + FT*(i)E]"FT*(i)  } 

+ EFT”(i)  ft  - E[I  + Ff"E]''''FT''''(i)} 

Disc rete  Transi tion  Matrix  of  Moving  Fixed  Span  for  Polynomial  Functions 


(96) 


When  the  row  vectors  of  the  T matrix  are  polynomials  as  considered  in  this 
paper,  then  the  updated  fixed  size  matrix  becomes  very  simple  in  structure. 
The  three  cases  of  interest  are: 


t(i,  n)  n=0,  1,  2,  ...N 

t(k,  m)  m = -M,  . . .-1,  0,  1,  . 

t(l,  y)  Y = 0,  1,  1,  . . .N 


and  the  polynomial  basis  by  Eq.  (38),  Eq.  (41)  and  Eq.  (43)  Section  (5)  are 


^t(i,  n)  = ,^T(i,  n)  TyO)  D(ho) 

/t(k,  m)  = <T(k,  m)  TyO)  D(ho) 

/t(l,  y)  = <T(1,  y)  Tu(p)  D(ho) 


(97) 

(98) 

(99) 


The  non-dimensional  quantities  are 


^ T(i,  n)  = n Tu(i) 
T(k,  m)  = < m Tu(k) 
/ t(l,  y)  = < y Tu(l) 


(100) 

(101) 

(102) 


where 


y - fn  2 3 

^ n = (0,  n,  n , n , . 


2 3 

^m  = (1,  + m,  m , +m  , 
< > = (1.  -Y,  . 


(103) 


tjr  • 


^ -t;  * IN 


I 

« 


If  we  pack  the  N + 1 row  vector  of  Eq.  (97)  into  a column  as  the  index  n varies 

/ \ 

T(i)  = l/n  Tu(i)  TuO)  D(ho)  (104) 


(N+l)xd  / • 


or  by  Eq.  (123)  Sed.  (5), 


T(i)  = N T^(i)  T^O)  D(ho) 


(105) 


(N+l)xd  (N+l)xd 


where 


1 0 


0 ) 


1 1 


N = 

(N+l)xd 


2 2^  2^^'^, 

3 3^  3^”^i 

I 


(N+l)xd 


(106) 


The  psuedo  inverse  of  Eq.  (105)  with  full  rank  factor  is 
f''(i)  = D'\ho)  T^'^p)  T"\i)  N* 


(107) 


dx(N+l) 


dxN+1 


The  square  matrix  inverses  are  given  by  Eq.  (164)  Section  (4). 
The  rectangular  matrix  psuedo  inverse  of  Eq.  (107)  is 
n"  = (N^N)“^N^ 


dx(N+l)  dxd  dx(N+l) 

T 

The  Grammian  N N is  given  by  Eq.  (60)  Section  5 in  terms  of  the  sums  of  the 
power  of  the  natural  number.  The  inverse  for  a 3x3  matrix  is  given  by  Eq. 
(224)  Section  (5)  as 


(108) 


3N"  + 3N  + 2 

(n’^N)'^  = g^  -(12N  + 6) 


-(12N  + 6)  10 

4(2N  + l)(8N-3)  -60 


(109) 


N(N+1) 


60 

N(N+1) 


(N+l)(N+2)(N+3) 


(110) 


and 


= (A)'')  "i'r  ■ ■ ■ 

The  time  point  at  indices  i and  n is 
t(i,  n)  = + (i+n)]ho 

where 

= to/ho 

and  the  powers 

<(  t(i,  n)  = <n  T^(i)  T^(p)  D(ho) 

The  time  point  at  i + 1 and  n is 
t(i+l,  n)  = [p^  + (i+l+n)]ho 
and  the  powers  by  Eq.  (154)  appendix  (A) 

< t(i+l,  n)  = <^n  T^(i)  T^(l)  T^(p)  D(ho) 
using  the  commutative  property 


T^(l)  T^(p)  = T^(p)  T^(l) 


we  have 


Vn 


’l  1 1 1 ...  1 1 


2 3 . 

2 2 
2 3 


(n'^n)' 


.r 


<(  t(i+l,  n)  = <n  T^(i)  T^(P)  D(ho)  O'^ho)  T^^d)  D(ho) 
t(i+l,  n)  t(i,  n)  ♦^.(i+l,  i) 


(111) 


(112) 


(113) 


(114) 

(115) 

(116) 

(117) 

(118) 

(119) 

(120) 

(121) 

(122) 
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<l>j.(i+l,  i)  = D *(ho)  T^(l)  D(hO)  (12 

dxd 

Packaging  the  row  vector  of  Eq.  (118)  in  a column 

T(i)  = N T^^(i)  T^^(p)  D(ho)  (12. 

(N+l)xd  (N+l)xd 

and  packaging  Eq.  (122) 

T(i+1)  = T(i)  4>  (i+1,  i)  (12: 

(N+l)xd  (N+l)xd  dxd 

and 

T*(i+1)  = <l>’^(i+l,  i)  T*(i)  (12( 

dx(N+l)  dxd  dx(N+l) 

Thus  the  transition  matrix  <t>^  has  inverse. 

<|)"^  = D"^(ho)  T'^(1)  D(ho)  (12 

The  new  projector  is 

T(i+1)  T"(i+1)  = T(i)  T*(i)  = NN*  (12i 

(N+l)xd  dx(N+l)  (N+1)(N+1) 


It  is  interesting  to  verify  these  results  with  some  simple  examples.  Consider 
a second  degree  polynomial  with  N = 3,  or 

d = 3 
N+1  = 4 


and  for  simplicity  let  P = ho  = 1 than 


T(i)  = N T.(i) 


(12' 


or 


• 

f 

((i+i)°,  (i+i),  (i+D^I 

10  0 

T^(i)  = 

I(i+i)°,  (i+i),  (i+i)^] 

= 1 1 1 

4x3 

((i+2)°,  (i+2),  (i+2)^] 

2 

12  2 

1 

i 

! 

i 

[(i+3)°,  (i+3),  (i+3)^] 

2 

) 3 3^ 

1 

1 

1 

* _ 

1 L 

0 

0 


0 

0 


,2 

1 


1 

= i 

1 

i+1 

i^+2i+l 

1 

i+2 

i^+4i+4 

1 

i+3 

i^+6i+4 

at  time 

point  i+1 

r ca+i+o) 

- 

(i+0)“l 

T(i+1) 

= 

<(1+1+1) 

T (i)  = 

< (i+1) 

<a+i+2) 

u 

< (i+2) 

<i+l+3) 

. '^(i+3). 

0 0 

1 

.2 

1 1 

: r 1 1 

1 i 

1 

1 1 

0 

1 2i 

0 1 

1 

2 

2 2^ 

0 

0 1 

0 0 

2 

1 

3 3 

r 

■ 

- 

■ 

2 

, 1 

0 0 

1 1+i 

l+2i+i'‘ 

I 1 

1 1 

0 1 

2(i+l) 

2 

1 1 


(■ 

1 

! 

1 

i 1 

1+i 

l+2i+i^ 

t 

1 

i+2 

i^+4i+4 

f 

j 

1 

i+3 

i^+6i+9 

f 

1 

1 

1 

i+4 

i^+8i+16 

T (i+1) 
u 


.2 

1 


1 i 
0 1 2i 


(130) 


N T (i)T  (1) 
n u 


(131) 


and  advancing  k by  one  point 

<(t(k+l,  tn)  = TJk)  TJI)  T^(p)  D(ho) 

By  Figure  (1)  Section  (1)  we  see  the  time  points  are  related 


(134) 


T(i)  - T(k) 
(W-l)xd  (N+l)xd 

The  time  point  is 


t(k,m)  = [8^  + (k  + m)]ho 
and  at  k+1 


t(k+i),  m)  =•  [6  + (k+1  + m)]ho 

o — 

where  by  Eq.  (66)  Section  (5) 


-(M-1)  (M-1)' 


M = 

(N+1 ) xd 


By  Equations  (35)  and  (43)  Section  (5) 

<:^(k+l,  m)  = <m  T (k)  T (1)  T (B)  D(ho) 
^ \ u u u 

and  packagewise 

T(k+1)  = M T^(k)  T^(l)  T^(B)  D(ho) 
(N+l)xd  (N+l)xd 

and  also  for  Eq.  (133)  the  matrix  is 


T(k)  = M T^(k)  T^(6)  D(ho) 
(N+l)xd 


(137) 


(138) 


(139) 


(140) 


(141) 


(142) 


As  before  commuting  in  Eq.  (140) 


T (k)  T (1)  = T (1)  T (k) 
u u u u 


(143) 


we  obtain 


T(k+1)  - T(k)  4'^(k+l,  k) 


(144) 
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(145) 


where  the  transition  matrix  is 

k)  = D"^(ho)  T^(l)  D(ho)  = ♦^.(i+l,  i) 

and  the  psuedo  inverse  is 
T*(k+1)  » <t)"^T*(k) 

By  Eq.  (142) 

T*(k)  = D'^(ho)  t'/(P)  T;;^(k)  M* 

“ dx(N+l) 


(146) 


(147) 


And  the  psuedo  inverse  of  the  rectangular  matrix  M is 

* T - 1 T 
M = (M^M)  M 

d(N+l)  dxd 


(148) 


The  Grammian  (m'^M)  is  given  by  Eq.  (119)  and  the  inverse  by  Eq.  (120)  Section  (5) 


The  projectors  are 

T"(k+1)  T(k+1)  = T*(k)  T(k)  = I 
dxd  dxd  dxd 


(149) 


and 


T(k+1)  T''(k+1)  = T(k)  T*(k)  = MM*  (150) 

(N+1)  (N+1)  (N+1)  (N+1)  (N+1)(N+1) 

Front  of  Span 

This  section  considers  the  T matrix  when  the  indices  are  referenced  to  the  time 
point  Jl  at  the  front  or  leading  point  of  the  span  and  the  index  y runs  from 
0 to  N.  By  Eq.  (99) 

<t(t,y)  = <(y  T^(^)  T^(P)  D(ho)  (151) 

and  advancing  I by  one  time  point 

<t(ii  1,  y)  = </y  T^()l)  T^(p)  T^(l)  D(ho)  (152) 

where  by  Eq.  (44)  Sec.  (5) 

<Y  = (1.  -Y,  Y^  V.  . . .) 

A vector  with  alternating  signs. 

The  matrix  of  N+1  vectors  of  Eq,  (151)  Is 


(153) 


(1 


T(l)  = r T^a)  T^(P)  D(ho) 

(N+l)xd  (N+l)(d) 

and  advancing  the  front  point  u by  1 

1(1+1)  = r T (1)  T (P)  T (1)  D(ho)  (1 

(N+l)xd  u u u 

or  in  terms  of  the  transition  matrix 

T(a+i)  = T(t)  <J>j.(Ji+i,  i-)  (i; 

where  as  for  the  two  previous  cases  the  transition  matrix  is 

1)  = D‘\ho)  T^(l)  D(ho)  (i: 

and  for  example  for  a 4 x 4 matrix  one  can  obtain  similar  results  to  equation 


SECTION  14 

Recurslve-Unconstralned-Unweighted  Parameter  Estlmatioa 

This  section  considers  the  Infinite  memory  case  as  opposed  to  the  fixed  memory 
or  recursive  fixed  span  cases  considered  later.  From  Eq.  (13)  sec  (10) 


- F ^ + v>j  - Fa^^  + 


with  estimate  of  the  parameters  given  by  Eq.  (33)  sec  (10) 

-N.  _ T?*  _ /-pT-rix”! 


a>j  .F*z>j  . (F^F)-^  f"z>, 


Assume  now  that  another  measurement  Is  available  z(k+l)  such  that  the  vector  of 
measurements  becomes 


\ 

Vz  (k+1)  / 


(k+^ 


where 


z (k+1)  = (k+1)  ^ + V (fc+1) 

- (k+1)  a (k+1^  + 'z  (k+1) 


we  now  seek  a new  parameter  vector  a(k+l)]>  determined  by  all  of  the  data  points 
of  Eq.  (3)  and  such  that 


<f(l) 


a (k+1)^  + I (k+1^ 


<4  (k+1) 


Since  this  Is  the  unweighted  case  the  sequence  j will  be  omitted  as  a subscript 
quite  often.  Eq.  (5)  Is 

zy  = F a(k+l)^  + z(k+l^  (6) 

(k+l)xd 


Multiply  Eq.  (6)  on  the  left  by 

T -1  T 
F*  = (F  F)  ^ F 

dx(k+l)  d(k+l)d  dx(k+l) 

The  psuedo-lnverse  Is  given  by  sec  (B)  Eq.  (9)  as 


dx(Ll) 


F*  - f X -Cl. 


1+  (F^F)"^ 


(F^F)'^ 


1+  (F^F)"^  J 

By  Eq.  (6)  and  Eq.  (7) 

a (k+1^  « F*  ^ 

dx(k+l) 

which  in  partitioned  form  becomes  by  Eq.  (3)  and  Eq.  (8) 


2>  - 


(f'^f)~^  <d  f*  2 (ib 

1+  (F^F)"^ 


(f’^F)'^  f>  2(k+l) 
1+  <J  (F^F)"^  fP> 


By  Eq.  (2) 


<^f  F*  2 a , 


but  the  predicted  value  of  the  measurement  at  k+1  based  on  the  parameter  at  k is 


2 (k+1,  k)  » <^f  (k+1)  a (^ 

Using  Eq.  (12)  in  Eq.  (10)  with  Eq.  (2) 

a (k+1^  = a (k^  + w (k+1^  z (k+1,  k) 


2 (k+1,  k)  “ 2 (k+1)  - 2 (k+1,  k) 
and  the  weight  vector  is 


w(k+lj^ 


(F^F)~^  f(k+l) 


1+  <^f(k+l)  (F^F)'^  f (k+^ 


when  F is  the  matrix  for  the  discrete  polynomial  fitting  functions  from 
Eq.  (124)  sec  (13) 

F » N T 

u0ho 

kxd  kxd  dxd 


TBinirr  irT 
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F* 

dxk 


n* 

u3ho 


with 


N* 


T -1  T 
(N  N)  N 


where  for  the  second  degree  polynomial 


3x(k+l) 


1111., 

0 1 23,., 

012  3 ., 


1 

^2 


and  for  the  next  time  point 


N 

3x(k+2) 


1 

1 

1 


1 

2 


1 

k 


1 

ld-1 

(k+D^J 


The  Grammian  of  Eq.  (16) 
nX 


T 

F^F 

d(k+l)d 


^uSho  (*> 


u6ho 


and 


■ ^6  ^uB 


also 

£<k+l)  (k+1)  (d> 

and  by  Eq.  (23)  and  Eq.  (22) 

(F^F)“^  £(k+l)^  = (k+1^ 


The  inner-product  term  in  the  denomonator  of  Eq.  (15)  can  be  written  as 
<f(k+l)  (f'^F)"^  f(k+l)>  = <(k+l)  (N^N)'^  (k^■l)> 

By  Eq.  (395)  sec  (1) 

T -1  -IT 

(N  N)  =*  B 

g gg  g 

and  in  terms  of  the  notation  change 

(N^N)"^  - Bg(k)  Bg^(k) 


(17) 

(18) 

(19) 

(20) 

(21) 

(22) 

(23) 

(24) 

(25) 

(26) 

(27) 
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with 


where 


Bg(k)  - 0 


6 

k(k-l) 


3k 

(k+1) (k+2) 


5k(k-l) 

(k+3) (k+2) (k+1) 


The  inner  product  of  Eq.  (25)  can  be  computed  as 


<Jk+l)  (N^N)"^  (k+1^  - <(k+l)  Bj 


we  note  also  that 


[i,  k+1,  (k+1)^]  -<k  r^(i) 


T (1) 
u 


hence 


<Jk+l)  (N^N)"^  (k+l)>  - T^)l)  (n'^N)' 


By  tedious  attention  to  detail  one  can  show 


<:^(fc+l)  (n'^N)"^  (fc+l)>- 
go  [2N^  + (61 - 103N^  - 150N  - 


The  weighting  vector  of  Eq.  (15)  can  easily  be  J 
polynomial  cases  over  the  infinite  span  for  exp< 
polynomials  are  orthogonal  over  a finite  discrel 
fnr  fixed  snan  filters. 
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Section  15  RECURSIVE  UNCONSTRAINED  UNWEIGHTED  STATE  ESTIMATION 
By  Eq.  (32)  and  Eq.  (33)  of  section  (12) 
z (N+^  = x(N)  (3>g  + V (IH-^ 

= x(N,  N)  (J>g  + ;(N,  N)  (N+^ 

where  by  Eq.  (34) 

x(N,  N)  (J>g  = F*^  z (N+J> 

and 

-1  -N 
F = FF  (0)  t 
zx  V 


F*  = 'I'  F (0)  F* 
zx  V 


Consider  an  additional  measurement 


N 

/-N 

o 

1 

z (N+i> 

!<“  I- 

1 

• 

1 

z(N)  1 

• 

z(N+l)! 

— J 

z(N+l) 

z(N+l)  = x(N+l)  + v(N+l) 


x(N+l)  = x(N+l)  (3>g 


x(N+l)  ()>g  = $ x(N)  (^g 


x(N)|>g  “ 4-“^  x(N+l)  (|>g 


Using  Eq.  (1)  and  Eq.  (12)  sec  (U?) 


1 


x(N+^  - FF^"^(0)  x(N+l) 

- F^^r-  x(N+l)  ip^ 

X (N+^  - FF^“^(0)  x(N+l)  (|>g 


By  Eq.  (12)  in  Eq.  (6) 

1 [ff  -1(0)  ♦-<»*«  x(»+l)  (3>. 


jz  (N+b 

j 

z (N-l-l)  j 


!<^)ex(N-n) 

'ff  -^o) 

V 


e 


x(N+l)  (p 


+ V (N+2 


+ V (N-^2 
s 


or 


z (N+^yfjx  (bs  ■**  '' 

W>e  / 

The  estimate  equation  is 

z (N-f-2>=/F^^  X (N-Hl,  N+1)  + ~z  (N-f2> 

V4)e  ) 

z(N-l-2^=  F^^g  X (N-l-l,  N+1)  (3)>g  + z (N+2^ 

where 

F ,,  = / F 
zxE  / zx  \ 

X 

The  psuedo-inverse  of  Eq.  (19)  is  given  by  Eq.  (9)  sec  (13)  as 


F* 

zxE 


»<^L^x>'^  e (>i 

1 +<)et<Fi  F )-i  tie<3>i 
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zL?CJ. 


(11> 

(12) 

(13) 

(14) 

(15) 

(16) 

(17) 

(18) 

(19) 


(20) 


The  unweighted  estimate  of  the  states  based  upon  N+1  measurements  Is  by  Eq.  (20) 
in  Eq.  (18) 


X (N+1,  N+1) 


F*  „ z 
zxE 


(N+2^ 


(21) 


or 


X (N+1,  N+1)  = 


e>, 

1+  4^  e>i 


(22) 


The  first  term  on  the  right  hand  side  of  Eq.  (22)  is  by  Eq.  (3) 

$F*^  2 (N+^  = x(N,  N)  (3>g  (23) 


The  scalar  element  in  the  second  term  of  Eq.  (22)  is 


^)  e «>F*  z(N+l)  = tx  (N,  N)  3(> 

(24) 

= ^3)  e X (N+1,  N) 

(25) 

= X (N+1,  N) 

(26) 

where 

$x  (N,  N)  p^  = X (N+1,  N)  p^ 

(27) 

is  the  predicted  value  of  the  state  estimates  at  time  N+1,  based  on  N measure- 
ments. The  position  level  term  is  the  first  coordinate  of  the  state  vector, 
that  is 


X (N+1,  N)  = X (N+1,  N)  (p^  = (1,  0,  0)  /x  (N+1,  N) ' 


X (N+1,  N) 
^x  (N+1,  N)- 


(28) 


Since  the  matrix  relating  the  predicted  measurement  to  the  predicted  state  is 
z (N+1,  N)  = ^<3)  e X (N+1,  N)  (p^  = X (N+1,  N)  (29) 

we  have  using  Eq.  (29)  in  Eq.  (23)  and  Eq.  (27)  in  Eq.  (23) 


X (N+1,  N+1)  ip^  - X (N+1,  N)  (p^  + w (N+1)  p i (N+1,  N) 


(30) 


5Q0 


L i 


where 


i (N+1,  N)  = z (N+1)  - z (N+1,  N) 
and  the  weighting  vector  Is 


w(N+l)  (3 


^ = 


F ) 0 

ZX  ZX  ^ 

1+  kj)  e F )"^  ^ e]> 

X ZX  ZX  1 


(31) 


(32) 


One  can  proceed  to  get  an  update  on  the  w vector  for  the  next  measurement  as 
well  as  an  update  on  etc.  One  must  also  obtain  the  variance  of  the 
predicted  states  as  weli^as  of  the  corrected  estimate  of  the  states. 


These  relations  are  derived  under  the  discrete  Kalman  filter. 
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Section  16  UNWEIGHTED  RECURSIVE  ESTIMATION  OVER  FIXED  SPAN. 


Consider  a polynomial  signal 

x(t)  = ^^ta  0>+  C^ta  ^ 

where 

<^t  = (1,  t, 

and 

f d d+1  d+0(\ 

<^t  = (t,t...t  ) 

where  a is  finite  or  infinite.  One  can  approximate  x(t)  as 
x(t)  = x(t)  + x(t) 


( 


(' 


where 


x(t) 


= <^ta^ 


The  discreet  vector  (for  N+1)  points  representation  of  x(t)  is  given  by 
Eq.  (124)  Section  (5)  as 


x(i+o) 

x(i+l) 


[x(i+N)  J 


<t(i+0) 


i(i,w  ) ^ = T(i)  a(i,w'\ 

^ ^ w xd 
P 


(t> 


<^(i+N) 


In  this  section  recursive  relations  over  a fixed  span  of  size  N+1  = w will 
be  developed.  For  a given  time  point  i,  the  index  n will  run  from  0 ?o  N, 
then  i will  advance  to  i+1  and  n will  repeat. 


One  can  express  this  relation  matrix-wise  as  (the  hat  symbols  over  the  x are 
ommitted  thus  Eq.  (14) 


and  for  the  next  set  of  data 


x(i+l^  = 

1 x(i+l+0)v 
[ x(i+l+l)l 

= 

Ix(i+1) 

x(i+2) 

[ x(i+l+N)j 

[x(i+N+l), 

the  shift-up 

and  out  matrix 

1 0 . . 

. 0 *y 

1 ° 

0 1 . . 

. . 0 

\ 

" (o 

0 . . . . 

. 1 

\o 

0 . . . . 

. 0 / 

f 

= S x(i 
uo 


x(i+N+l)  (8) 


and 

e = (0,  0 ...  1) 

Also  for  a span  indexed  with  respect  to  a midpoint  we  have 


(9) 


(10) 


x(k-M) 

1 0 

^x(i+0)  ~1 

x(k-M+l) 

0 1 

x(i+l) 

x(k-l) 

_ 

i 

x( i+M) 

x(k+0) 

1 

• 

x(k+l) 

° i 

• 

• 

0 1 ! 

• 

• 

i 

1 

x(i+N) 

• 

J 

- 

x(k+M) 

and  advancing  k by  one  point 


x(k+l-M) 

x(k-M) 

x(k+l+0) 

= S 

uo 

x(k+0) 

x(k+l) 

+ G x(k+l+M) 

x(k+l+M) 

x(k+M) 

(11) 


(12) 


and  likewise  the  connection  between  a midpoint  span  and  a front  point  span  is 


■x(k-M)  ■ 

' x(y-N) 

x(k-M+l) 

• 

x(y-N+1) 

• 

x(k+0) 

— 

« 

x(y-M) 

x(k+M) 

x(y-O) 

and  advancing  the  span 


■x(y+1-N)- 

* 

x(y-O) 
-x(y+1)  - 


x(y+1) 


The  following  time  relations  will  be  used 
4(i,n)  = <T(i,n)  T^(p)  D(ho) 

<t(k,m)  = 4<^k,m)  T^(p)  D(ho) 

4(1, Y)  = T^(P)  D(ho) 

and 


<T(i,n)  = 4 

4(k,m)  = 

4(1, Y)  = ^ T^(l) 

and 

y - 23  d-1) 

<01  -(l,n,n,n  ...n 

4 “ ( 1 , I™ , • • • ) 

<^  = (i,  -y,  Y^,  -Y^  . . •) 

If  we  package  the  N+1  measurements  as  a function  of  i,  we  have 


'z(i+0)' 

z(i+l) 


,z(i+N)/ 


(i: 


(1^ 


(If 


(U 


(1- 


(1« 
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1 


or  as  a function  of  k 


z(k-M) 


2(k+M) 


= T(k)  a(k;  wp^  + z(k,  wp^ 
(N+l)xd  ^ 


and  as  a function  of  the  span  front  index. 

z(£)  (w^  - T(£)  a(£;  wp)^  + z(£;  wp^ 

X (N+l)xd  ^ 

if  we  advance  the  span  one  point  Eq.  (18)  becomes 


z(i+l) 

z(i+l) 


z(i+N+l) 


= T(i+1)  a(i+l;  wp^  + z(i+l^ 


and  similarly  for  Eq.  (19)  and  Eq.  (20), 


z(i+i)  = s^^z(i^  + e 


z(i+N+l) 


and  by  Eq.  (89)  Section  (13) 

T(i+1)  = T(i)  +e  ^><^(i+N+l) 
(N+l)xd  (N+l)x(N+l)  ^^+1 


t(i+l) 

t(i+2) 


\t(i+N+l)/ 

The  unweighted  solutions  to  Equations  (18,  19,  20)  requires  computing  the 
psuedo  inverses 

T*(i)  = (T^(i)  T(i))"V(i) 
dx(N+l)  d(N+l)d 

Jl. 

and  similar  for  T (k),  and  T (£) . 

The  update  solution  of  Eq.  (21)  requires 
T''’(i+1)  = (T^(i+1)  T(i+l)]‘^T^(i+l) 

These  relations  have  been  derived  in  Section  (13). 

Partitioning  the  measurement  vector 
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then 


(-1>) 


^<^o\  a(i)  ^ + z(i^ 
I Tn) 


(27) 


and  the  psuedo  inverse  of  the  partitioned  matrix  by  Eq.  (71  - 72)  Section  (13) 
yields 


i(i)^  = (t'^T  )'^t>  z + [T*  - (t"^!  )'^t><t  T*]  z»i> 
X n n o n n n ^o\  n 


(28) 


advancing  the  time  point  to  i+1 

/z^  \ = 

\z(i+N+l)/  L„.., 


Tn  \ a(i+l^  + z(i+l^ 


<t 

N+1, 


and  using  the  psuedo  inverse  relations  of  Eq.  (84  - 85)  Section  (13) 
= IT*  - <1  z|>  T (T;T„)-t>,„ 

\+l  \+l 

Subtracting  Eq.  (28)  from  Eq.  (30) 

V ''  V T -t  N-bl  ■>  V 

a(lTl)>  = a(i)>  * (TJ_)  ^ 

AN+1 


Eq.  (31)  can  be  rewritten  as 


= a(i)^  a ^ 

AN+1 

* 1 


Ao  \+l 

Another  expression  for  updating  a(i+l)^^  can  be  obtained  from  Eq.  (22) 


s z(i^  + e> 

UO  y a^N 


N+l^N+1  ■ T(i+1)  a(i+l)  + z(i+l)]^ 


and  by  Eq.  (27) 

^N+l^N+1  ^ T(i+1)  a(i+l)>+  z(i+l^ 

JL 

Operating  on  (34)  by  T (i+1) 

a(i+l^  = T*(i+l)S^J(i)a(i)  ^ + T*(i+1)^^^2 


(29) 


(30) 


(31) 


(32) 


"N+1 


(33) 


(34) 


(35) 
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since 


T*(i)z(i^  = T*(i+l)S(i+lJ>  = ^ 
and  by  Eq.  ( ) 

T*(i+1)  = <l>^^T*(i) 

hence 

T*(i+l)z(iJ>=  <t)‘^T*(i)z(i^  = o)> 

By  Eq.  (28) 

a(i^  = T (i)z(i^ 

and 

z(i+N+l,  N+1)  = <t  a(i^  = <t  T"(i)z(i)^ 
where  the  hat  symbol  is  the  predicted  measurement. 


By  Eq.  (86)  Section  (13) 

\+l 

and  adding  and  subtracting  to  Eq.  (35)  Eq.  (39)  t .,es  Eq.  (40) 

a(i+l)>  = T*(i+l)S^^J(i)a(i)>  + N+1) 


\+] 


+ (TTT^)'lt>j,^^z(i+N+l;  N+1) 

\+l 

Using  Eq.  (39)  in  the  last  term  of  Eq.  (41) 

^-l. 


N+-L 


.(i+l)>=  [T''(i+l)S^J(i)  + a(i)> 

\+l 

+ (T^T^)'^>n+i  2(i+N+l;  N+1) 


now 


+ 1 


/<f\  = l\ 

\ w 
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i 


and  by  Eq.  (86)  Section  (13) 

T*(i+1)  = [T*  - (T^T  % T*,  (t'^T  )'^^„  J 

dx(N+l)  ^ N+l\  n’  n n'^  )^N+1^ 


T (i+1)  S T(i) 
uo 


dx(N+l)(N+l)(N+l)(N+l)xd  dxn 


\+l 

[t”  - (t'^^T  )’^^„^,<:t  T*]T 
n n n n'^  n 


AN+1 


and 


T"!  = I 

n n 

dxd  dxd 

and  (46)  and  (45)  in  the  multiplicative  matrix  term  of  a(i^  in  Ea  (42) 
yields 

T(i)  ♦ = I 

\+l 

and  Eq.  (47)  in  (42) 

a(i+l)>=  a(i)>+  (T^T^)”^t>j^+lZ(i+N+l;  N+1) 

\+l 

a(i+l^=  a(i)^+  w(i+l ) ^ z(i+N+l ; N+1) 


1 • <^t(TjT 


where 


Section  17  CONSTRAINED  LEAST-SQUARES  RECURSIVE  ESTIMATION  FOR  ONE  DEGREE  OF 
FREEDOM 


This  section  considers  an  increasing  batch  of  data  being  fitted  to  a d-1 
degree  polynomial,  or  a d dimensional  vector  of  parameters  where  the  first 
d-1  parameters  are  constrained  to  be  the  initial  estimates,  thus  one  has 
a scalar  recursive  relation.  For  k measurements  partitioned  as  shown 


or 


IT,  t(J>] 

(kxd-1)  d 


( 


= T a(k)(d-l^  + t(k^  a(k)^  + z(^  ( 

kx(d-l)  ' 


It  is  assumed  that  d-1  of  the  parameters 


a(k)(d-l^  = a(k-l)(d-l^ 


a 


( 


is  known  from  previous  stage,  then 


z(k)  - T a(d-l > 
kx(d-l) 


t(k)d  a^(k) 


( 


Multiplying  by  the  psuedo-inverse 

^<|'k)t  fz(k\  - T a(d-lS]  = a,(k), 

where 


d<k)t  t(k>^ 


= 0 


If  we  now  have  k+1  observations 


z(k^  = . T t 

^k+1  j(k+l)x(d-l) 


z(k+l^  - T a(d-l  ^ = t(k+lX  a(k+l),  + z(k+l^ 
/(k+l)xd-l  / 'd  d / 


( 


( 


( 


or 


a(k+l)^  = ^k+l)t  [: 

^<k+l)tt(k+i;^ 


A 

(k+l)  - T a(d-A 

(k+l)x(d-l)  ' 


( 


The  partitioned  matrix  is 


T = 

(k+l)xd 


T 

kxd-1 


l<^-l)t 


k:r 


' T 
kxd 


<)t 

k+1 


[T,  t(k+b  ] 

(k+i)(d-n  ** 


or 


/ T N 
, kxd-1  ' 
T = ( 1 

(k+l)x(d-l)  \<4-l)t  ' 

\ k+1 


and 


t(k+l 


K - 


t(kN 


Xd 


'k+l,d  , 


and 


*(k*l)'  = d<«“'>d  ♦ ‘Ll,, 


^k  H+l,d 


Using  Eq.  (II)  and  (12)  in  Eq.  (9) 

, =±  (<k] 
g(k+l)  d) 


H*i,d>  '■'i  - L I •('*->>  ' 


kxd-1 


'k+1 


\ <d-l)ta(d-]> 
k+1  ^ 


1 


' 8(k+l) 
+ t. 


<k)t(z(ls>  - Ta(d-i!j 
d kxd-1 


I 


k+l,d^‘'k+l  " k+^"^^*^  a(d-^)| 

The  predicted  value  of  the  measurement  is 


Kkkl;  k)  = ;(k)(^  • ( „) 


i(kti,k)=  ;(d->  *t  Ick) 


k+l,d“d' 

Adding  and  subtracting  Eq.  (16)  to  last  term  of  Eq.  (15) 


a(k+l)  = 1 

^ g(k+l) 


j4)t(z(^  kkCd-ir"*^  * 'kki,d‘vi  - k(k+i,k) 
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'■■I  ^ 


By  Eq.  (5)  in  (17) 


i(k+l)^  = a(k)j  ( ‘*<k)tt(^^  + + z(k+l;  k)  t 


k-H,d 

^(k+1) 


a(k+l)d  = a(k)^  + ^ z(k+l,  k)  (19) 

t(k+l)><^t(k+l) 

which  is  a scalar  recursive  equation. 

Constrained  Least  Squares  Recursive  Estimation  for  Multiple  Degrees  of  Freedom . 

This  section  partitions  the  unknown  d dimensional  a(d^  vector  into  two  sub- 
vectors of  dimensions  d^  and  thus  for  k measurements 


z(^  = [T  , T] 
kxd  kxd. 


/ a(k)(d-^'\  + z(^ 

Va(k)(d2>;  / 


where  the  constraint  is  imposed 

a(k)(dj\  =a(k-l)(dj)  “ } 


dj  + = d 


z(k^  - T a(d^  ^ = T a(k)(d2'^  + z(k\ 
kxd^  kxci^  ^ ^ 

Multiplying  Eq.  (23)  by  the  psuedo  inverse  of  T 

kxd^ 

T*  (z(k)  - T a(d  \ ] = a(k)(d_) 
d^xk  ^ kxdj  ' ^ 

The  predicted  scalar  measurement  at  the  next  stage  is 

z(k+l;  k)  = ^d)t  a(k)(d^  = [^t,  / a(dj^  \ 

k+1  ^+1  1 

a(k)(d-  j 

/ - \ / 
z(k+l;  k)  = ^dj)t  a(dj^  + ^d2)t  a(k)(d2^ 


The  partitioning  for  k+1  measurements  is 


T 

kxd 


'k+1 


1 


kxd. 


\ 


also 


! ^d^t, 

\ 


k+1 


k+2 


I 


+ z 


(k+1^ 


f zCk^'' 


\^^k+l 


T 

(K+l(xd 


1 


T 

(k+l)xd. 


z(k+l) 


or 


(k+l)  - T a(d.\  = T a(k+l(d  ^ + z(k+l  > 

^(k+l)xd,  Vfk+lWd  ^ 


(k+l)xd 


'1 

Multiplying  Eq.  (41)  by  the  psuedo  inverse  as  shown 

t” 


d2x(k+l) 


[z(k+l^  - T a(d^]  = a(k+l)(d2) 
(k+l)xdj  * ' 


By  Eq.  (39) 


T 

(k+l)xd„ 


.*■'  T \ 
I kxd 


2 

k+l/ 


and 


(d2x(k+l) 


(T^ 


T) 


-1 


d2(k+l)(k+l)xd2  d2x(k+l) 


The  Grammian  is 


[t'*’,  t(d2)] 


r, 


(d2xk) 


k+1 


T 

kxd,. 


T^T  + t(d^d2)t 
k+1 


d2(k)d2 


i V| 

1 k+1  I 


By  the  Householder  inversion  lemma,  the  Grammian  inverse  is 


(T^T)"^\=  (T^T)"^  U - t(d2\(d2)t  (T^T)“M 

» j/'i. 


d2(k+l)d2j  d2(k)d2 


d2(k)d2 


Ad2 


A.,,  = 1 + /d2t  (T^T)"^t(d2) 


k+1  d2(k)d2 


k+1 


(39) 


(40) 


(41) 


(42) 


(43) 


(44) 


(45) 


(46) 


(47) 


J 


]' 
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Carrying  out  the  analysis  one  obtains 


T"(k+1)  = T''(k)  - (T^T)"^t(d2){d2)t  T*(k)  , (T^T)"^t(d|j 

d2(k+l)  d^xk  d^(k)d„  k+1  d„xk  d„(k)d„  


d2xk 


d2(k)d2 


d2xk  d2(k)d2 


d2 


d2 


(48) 


d2xk 

The  first  term  on  the  left  side  of  Eq.  (42)  is 


t"  z(k+lV  = T''(k)z(k)  - (T^T)‘^t(d2>  (d2)t  T*(k)  z(k) 

W+1^  d2xk  b-4-1  A • 


d2(k+l) 


k+1 

^d2 


d2xk 


(49) 


+ (T’^T)'^t(d2>.  , 1 z 

j / 1_  \ j K'f  1 . 


d2(k)d2 


k+1 


"d2 


Consider  the  second  matrix  product  term  of  Eq.  (42) 


d2x(k+l) (k+lxd^ 


T = [T''(k+1),  t''‘(d2)^^j] 


daxk 


; T ' 

kxdj  , 

dl)t 

k+1 


T'(k+1)T  + t"(d^  <fdl)t 
(d2xk)fkxdj)  k+1 


(50) 


and  by  Eq.  C48)  in  Eq.  (50) 

f'T  = [T''(k)T  - (T'^T)'^t(d2)  (d2)tT''(k)T  1,  (t’^’t)"^  t(d2>  <dlt]  (51) 

d2x(k+l)(k+l)xd^  d2(k)dxd^  d2(k)d2  k+1  d2xk  kxd^  A"2  d2(k)d2  ~^d2~ 

The  second  matrix  vector  term  on  the  left  of  Eq.  (42)  which  is  Eq.  (51)  times 
the  vector  a(dl^  is 

t“t  a(dl\  = t't  a(dl)  - (T^T)"^  t(d2^  ^2)t  x 


d2(k+l) (k+l)xdj 


d2xkkxdj 


d2(k)d2 


k+1 


d2 


(52) 


T"(k)  T a(dl  \ + (T^T)"^  t(d2\  {dl)t  a(dA 


(d„xk)  kxd 


1 


d2(k)d2  A^2 


By  Eq.  (25) 

<dl)t  a(dl^  = z(k+l;  k)  - <(d2)t  a(k)(d2^ 


k+1 


(53) 


k+1 
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^ I 


Using  Eq.  (53)  in  the  last  term  of  Eq.  (52) 


t"  T a(dl^  = t"  T a(dl)  - (ri)"^  t(d2>  {dl)t  T T a(dl) 
d2(k+l)  (k+l)xdj  d2xk  kxd^  d„(k)do  K+l  d„xk  kxd. 


d2(k)d2 


d2xk  kxdj 


d2 


^+_(T^T)"^  t(d2)  (z(k+l;  k)  - {d2)t  a(k)(dA) 

J |}j-1  ' 


^d2 


"k+l  k+l 

By  Eq.  (42)  using  Eq.  (49)  and  Eq.  (54) 


a(k+l)(d2)  = [T*(k)  - (t'’‘’t)"^  t(d^  (d2)t  T*(k)]z(kV 


d2xk 


d2(k)d2 


k+l 


d2Xk 

AdT 


+ (T  T) 


:(d2> 


k+l  "k+l 


d2(k)d2  Aj2 


[T 

d2xk 


(tTt)-I  ^^2)t  T"]  T a(dl} 

d2(k)d2  k+l d„k  kxd,  ' 


d^k  kxdj 


d2 


(tTt)-I  (z(k+l;  k)  - {d2)t  a(d2)) 

d.,(k)d„  k+l  k+l 


^d2 

Rearranging  Eq.  (55) 


a(k+l)(d2\  = T”(k)  [z(k')  - T a 

/ d-y'-  ' 


d2Xk 


kxd 


(dj)] 


1 


(tTt)-I  ,-(d^  (dl)t  T*(k)z(k\  1 
d2(k)d2  k+i  d2xk  ' ^d2 


T -1 

n m \ ^ 


+ (T^T)  * t(d2)  (d2)t  t"  T a(dl\  1 


d2(k)d2 


k+l 


d2xk  kxdj 


/r- 

d2 


+ (T^T)"^  t(d2)  ^d2)t  a(k)(d2\ 

J / 1.  \ J 1 A • 


d2(k)d2 


+ (T^T)“^  t(d2>  ?(k+l,k) 


k+l  A 


d2 


d2(k)d2-Tr^2 


The  first  term  by  Eq.  (56)  is  a(k)(d2^  hence 

a(k+l)(d2\  = a(k)(d2)  + 1 (t’^’t)"^  t(d2)  z(k+l,k) 
' ^dl  ‘*2^''^‘^2 
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r • r *•  »V‘  • * 


(54) 


(55) 


(56) 


(57) 


The  three  middle  terms  on  the  right  of  Eq.  (56)  equal  zero  since 

-(T^T)"^  A(d2)  {d2)t  T*(k)  z(k)j_ 

*^2**^2  ^2**^  ^d2 


+ (T^T)"^  t(d2}  {d2)t  T*  T a(dl)  _1_ 
d2xd2  k+1  d^xk  kxd^  Aj2 

+ (T^T)"^  t(d2)'  {d2)t  a(k)(d2\  = 
d2xd2  k+1  A^2 

_ 1 (T^T)"^  t(d2}  (d2)t  U-T*  z(k)  + T*  T a(dl)  ] 

A /i  ^ Ir  vtr  lrv/^  ' 


(51 


Ad2  d2xd2 


d2xk 


d2xk  kxdj 


+ a(k)(d2^ 
since  by  Eq.  (24) 


= 0 


> 


i(k)(d2>  = T*  (z(k\  - T a(dl)  ] 
d2xk  ' kxd^ 


(5! 


Thus  we  see  by  Eq.  (57)  we  need  only  estimate  d2  parameters  instead  of  d^  + d, 
and  the  recursive  relation  is 


a(k+l)(d2^  = a(k)(d2^  + W(k+l)(d2^  z(k+l),  k) 
where  the  weight  vector  is 


W(k+l)(d2S  = (T^T)"^  t(d2) 

/ J Z'  I,  ^ J — X 
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^d2 


and  by  Eq.  (47) 

A = 1 + (d2)t(T^T)"^  t(d2) 

1 J / I- \ J 


(6 


SECTION  18 

OPTIMAL  WEIGHTING  MATRIX  FOR  PARAMETER  ESTIMATION 


This  section  obtains  the  optimal  weighting  matrix  for  minimizing  the 
trace  of  the  variance  of  the  estimate  of  the  parameter  vector  given  by 
Eq  (89)  sec  (10)  as 


= w R w 

aa 


dxd  dxk  kxk  kxd 


subject  to  the  linear  constraint  of  Eq  (18)  sec  (10)  (R  and  F given) 


W F = I 


dxd  dxk  kxd 


we  saw  in  previous  sections  the  unweighted  solution  occurs  for 


W = F 


Minimize  trace  of  Eq  (1)  subject  linear  constraint  of  Eq  (2). 

The  minimum  variance  estimate  is  derived  in  most  test  books  using 
LaGrange  multiples.  A few  of  the  recent  psuedo-inverse  books  obtain  the 
solution  via  algebraic  utilization  of  the  psuedo-inverse.  These 
derivations  are  believed  to  be  original  and  appealing  to  geometrical 
intuition.  The  derivation  will  be  step-wise  starting  with  the  case  d=l. 
By  Eq  (1) 


=<(w  R w)> 


and  by  Eq  (2) 


i =<w  f^ 


where  for  the  polynomial  case 


<f  =<1  = (1,1,1,...). 


The  variance  matrix  R is  assummed  full  rank  (one  can  solve  the  non- 
full rank  case  but  it  will  not  be  presented  here).  Since  R is  full  rank 
and  symmetric  (positive  definite)  it  has  factors 


R = B B 


or  Eq  (7)  in  Eq  (4) 


O-  = ^w  B B^w  y 


where 


(9) 


<(y  = ^ B 

and 

= <;w 

Using  Eq  (11)  in  Eq  (5) 

= <y  B'"  = 4 


where 


■'  <=> 


(10) 


(11) 


(12) 


(13) 


we  see  the  problem  to  minimize  the  quadratic  form  in  the  unknown  vector  ^w 
of  Eq  (8)  subject  to  the  one  degree  of  freedom  linear  constraint  of  Eq  (12) 
can  now  be  stated  as  finding  the  vector of  minimum  magnitude  (norm)  which 
is  constrained  to  lie  on  the  hyperplane  or  Eq  (12). 

Multiply  Eq  (12)  by  the  psuedo-inverse  of  or 

Decompose  the  vector  /y  into  a component  along  c^and  one  perpendicular. 


that  is 


with 


and 


4 = (y  ^4 

4 =4  ^cc* 


(15) 


(16) 


(17) 


If  we  obtain  the  normal  form  of  the  equation  of  the  plane  of  Eq  (12) 

^o  ~/y  ^ (18) 

where  the  unit  normal  is 

^ 

f , Vi 


le  dist£ 


(19) 


we  know  that  the  distance  out  from  the  origin  along  the  normal  to  the  plane 
is  by  Eq  (18) 


d = «. 


<o^ 


/2 


(20) 
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The  picture  is  now  as  shown  in  Fig  (1) 


FIG  (1)  ORTHOGONAL  PROJECTION 

Clearly  (via  the  orthogonal  projection  theorem)  by  obvious  geometrical 
intuition  the  nearest  approach  to  the  origin  for  the  set  of  vectors 
constrained  to  terminate  on  the  plane  is  the  vector  along  the  normal,  or 

<y. 

By  Eq  (13) 


^c*  = /c_  = ^f  B 


(21) 


} B"^  B“^  f^ 


or 


</  = (fjZ- 


(22) 


^ (B  b”^)"^  f^ 


Using  Eq  (22)  in  Eq  (14) 


<?  = ‘o  <4 


,-T 


^f(BB^)”^  f ^ 

By  Eq  (23)  in  Eq  (11) 

<"  = ‘o 

^ f (BB^)"-^f^ 


(23) 


(24) 


or 


(fQ-  f> 


(25) 


when  <1^  = 1 and  ^ f = ^ 1 


which  is  the  well  known  solution  for  the  minimum  variance  sequence  of 
weights  to  estimate  a constant. 

Clearly  when  the  quadratic  surface  is  a hyperplane  Q = I and 

Og  = ■^  w (27 

which  is  the  equation  of  a family  of  spheres.  The  minimum  sphere  touches 
the  fixed  hyperplane  as  shown  in  Fig  (2) 


FIG  2 MINIMUM  SPHERE  TANGENT  TO  PLANE 


By  Eq  (27)  the  minimum  sphere  has 
2 

\ w w/  = r 

o 

By  Eq  (5)  the  normal  form  for  the  Equation  of  the  plane  is 

^ = {w  £} 


(28 


(29 


which  is  the  distance  to  the  plane  that  is 


r = Jl 
o o 


By  Eq  (24)  in  Eq  (28) 


2 

r = 
o 


(f 


2 _ 2 
o 


(3o: 


hence  Eq  (30)  and  (31)  agree.' 


(31 


Consider  next  the  case  where  the  parameter  vector  has  dimension  d, 
that  is  a d-1  degree  polynomial. 

By  Eq  (1) 

T 

tr  \ = wRw  (c 

da 

Partition  w intor  row  vectors 


tr  = Vw|  R[w)^...w^^] 


tr  = ^^wRw)^  +...+  ^{wRw)^ 


The  problem  is  now  to  minimize  Eq  (34)  subject  to  the  linear  constraints 
of  Eq  (2),  Using  the  full  rank  factors  of  Eq  (7)  in  Eq  (32) 

tr  ^:  = wAA^w^  = YY^  (35) 

= ^yy\  (36) 


where 


Y = wA 


w = YA 

Using  Eq  (38)  in  Eq  (2) 


L = YA  F = YC 
o 


dxk  kxd 


C = A"^  F 


Multiply  Eq  (39)  by  C 
i!e  * 

L C = YCC  = Y 
o 


v/here 


Y = Y + Y 
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and 


Y = (I-CC’‘)Y  (43) 

The  psuedo  inverse  of  Eq  (40)  is 

C*  = (C^C)"^  c'^  = (f'^A"^A“^F)”^  F^A"”^  (44) 

dxk 


or 

C*  = (f"^  R"^  F)"^  f'^  A'^  (45) 

Using  Eq  (45)  in  Eq  (41) 

Y = L c”  = L (f'^  R"^  F)"^  F^  A”^  (46) 

o o 

and  Eq  (46)  in  Eq  (38) 

w = L (F^  r"^  F)"^  f'^  R'^  (47) 

o 

SOLUTION  OF  MATRIX  CASE  USING  KRONECKER  MATRIX  PRODUCT . 

The  solution  to  the  matrix  case  can  be  done  using  the  Kronecker 
Tiftrix  product.  The  technique  requires  a simple  extension  to  the  vector 
case  where  d=l,  that  is  by  Eq  (4) 

o-  = ^wRw^  (48) 

and  assume  a vector  of  linear  constraints  instead  of  a matrix  as  in  Eq  (2), 
that  is 


= CwF 
o ^ o 

kxr 


(49) 


where  the  F of  Eq  (49)  is  not  to  be  confused  with  the  matrix  F of  Eq  (2) 
o ^ 

which  is  the  matrix  of  fitting  functions. 


Using  Eq  (11)  in  Eq  (49) 


■'1  = F C 

kxr 

Multiply  Eq  (50)  as  before  by  C 

* ^ ft 

=<;sr  B F^c  c c , 

where  as  before 

. A £* 

<y  =<^y  +\y 


with 


<^y  =<'yC  C" 


(50) 

(51) 

(52) 

(53) 
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'jtam 


Note  that  the  projector  CC  is  kxk  and  if  k=r  (that  is  if  there  are  k 
linear  constraints)  then  there  are  no  remaining  degrees  of  freedom  left 
and  (full  rank  C) 


<y  <y  = <1^  (54 

a constant  vector.  Equation  (54)  implies  that  the<  w vector  is  also  a 
constant,  hence  the  quadratic  of  equation  (48)  can  not  be  minimized  since 
it  is  also  a constant.  The  problems  of  interest  is  for  r^  k,  then 


C“  = (C^C)  ^ CT  (55) 

nxk  nxr 

c''  = (f'^  F )”^  f”^  B"'^  (56) 

o o o 

c"  = (f"^  (Bb'^)~^F  )~^  f'^  B“^  (57) 

o o o 


or  using  Eq  (57)  in  Eq  (51) 

<Cy  = [f'^  R”^  F ]”^  f'^  B"^  (58) 

and  Eq  (58)  in  Eq  (11) 

<'w  = <1  [f"^  R~^  F f"^  r'^  (59) 

The  Kronecker  product  enters  the  picture  via  Eq  (38) 

vec  w = vec(YA  ^)  = (A  '^3  I)  vec  Y (60) 

where  Eq  (49)  sec  ( G ) is  used. 

Also  by  Eq  (37)  and  Eq  (49)  sec  ( G ) 

T 

vec  Y = vec(wA)  = (A  3 I)  vec  w (61) 

we  use  the  trace  properties 

tr  Y y'^  = tr  y'^Y  = (vec  Y)'^  vec  Y (62) 


hence  the  inner  product  expression  of  Eq  (62) 

T '■  ■ — 

(vec  Y ) vec  Y =^.p)  y y (p,^^ 


(63) 


where 

P 

is  the  same 
operator  on 


= k + d 

form  as  the  problem  of  Eq  (48)  and  Eq  (50  wher  we  use  the  vec 
Eq  ( 39 ) , that  is 


(64) 


vec  L = vec(YC)  = (C  B I)vec  Y 
o 

2 

The  matrix  C by  Eq  (40)  is  k x d in  size  and  the  vec  L is  d x 1 in 
size  that  is 

vec  L = [C^  a I]  vec  Y (65) 

o 

2 

d xl  dxk  dxd  kdxl 


C a I = r 


d^xkd 


we  now  need  the  psuedo  inverse  of  T or 

r“  = [c’^^’a  i]‘‘  = (r  r"^)"^ 


By  Eq  (83)  sec  ( ) 

= (c'^®[)'^  = C(S>I  (68) 

Using  Eq  (68)  and  Eq  (66) 

rr'^  = (c'^  (»  I)(C  (g/  I)  (69) 

By  Eq  (54)  sec  ( ) 

= c'^OS)!  (70) 

The  inverse  of  the  Kronecker  product  of  Eq  (70)  is  given  by  Eq  (55)  sec  ( G ) 
as 

[C^C  a I]"^  = (c’^C)"^  0 I (71) 

Using  Eq  (71)  in  Eq  (67)  and  Eq  (54)  sec  ( G ) 

r"  = (cai)C(c^c)"^  (g)  I]  = c(c'^c)"^  a i (72) 

By  Eq  (44) 

* T _ 1 T 

C = (C  C)  C (73) 


or  transposing 

C*’^  = c(c'^c)“^ 

hence  using  Eq  (74)  in  Eq  (72) 
* ftT 

r = C a I 


Eq  (75)  in  Eq  (65) 

^ A 

vec  Y = r vec  L 

o 

= [C  a I]  vec  L 

o 

and  the  de-vec  or  unpack  operation  given  by  Eq  (49)  sec  (G  ) 

^ * 
vec  Y = vec  L C 
o 


or 


Y = L C 
o 

which  agrees  with  Eq  (41). 

3y  Eq  (13)  sec  (10) 

z)  = F a)+  v)  = F a)+  z) 
kxd 

and  premultiplying  Eq  (80)  by  w 

dxk 


(76) 

(77) 

(78) 

(79) 


(80) 


w z^  = a ^ (81) 

dxk 

One  can  multiply  Eq  (80)  by  a full  rank  matrix  M kxk  in  size  to  obtain 


Mz^=MFa^+Mz^ 

(82) 

kxk 

or 

MzV=F  a^+Mz^ 

’ m 

(83) 

with 

F = M F 

(84) 

m 

kxd 

Multiply  Eq  (83)  by  the  psuedo  inverse  of  Eq  (84) 

f"  M z ^ = f"  F a )■  = a V 
m f mm'  r 


The  psuedo-inverse  of  Eq  (84)  is 

f”  = (F^  F )“^  F^  = (f'^m'^MF)"^  f'^m'^ 
m mm  m 

Using  Eq  (86)  in  Eq  (85) 


(85) 


(86) 
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with  the  denominator 


d = r - •^rR  , 

and  the  cross  correlation  terms  are  in  the  vector 

For  the  assumption  of  no  serial  correlation,  that  is 
<(r  = <0 
Eq  (7)  becomes 

,-i 


(8) 


(9) 


(k+l)(k+l) 


L 0 


r-1^ 


(10) 


The  transpose  of  the  expanded  fitting  functions  matrix  of  Eq  (3)  is 
^ f(k+l)^  ] 


F 

dx(k+l) 


dxk 


and  the  product 

[F’^f(k+1)> 


] Fr'^ 

[<0  r-J 

T -1 

j _ j -^T 


= [F^R~^,f(k+l))>  r"^] 


(11) 


(12) 


as 


The  inverse  of  the  expanded  F R F matrix  is  given  by  Eq  (48)  sec(  13  ) 

(F^R'^F)"^  = (F^R”^F)"^[I  - f^f(F'^R~^F)r~^]  (13) 

d(k+l)d 


1 + r“^  <(f(F'^R~^F)"^f)> 


Multiplying  Eq  (12)  and  (13) 


W 

dx(k+l) 


W - ( F'^R~^F)~^^Xf ( 


:f'R  "F)  "F'R  r ", 


1 + r"^<(f(F^R"^F)"^f^ 

(F^R"V)‘^f)>  r~^  - (F^R~^F)~^f^<^f(F^R~V)~^f)^  1 

1 + r"^  <^f(F'^R‘^F)"^f^  J 

Multiplying  Eq  (14)  by  the  partitioned  measurement  vector 


(14) 


L5] 


Section  20  HOMOGENEOUS  DISCRETE  DYNAMIC  PROCESS 


Consider  a sequence  of  j vector  trajectories 
= <Kk+l,  k)  X(k)>j 

for  k = 0,  1,  2 ... 
j > k 

or  packagewise 

X(k+1)  = $(k+l,  k)  X(k) 
pxj  pxj 

where  $(k+l,  k)  is  the  usual  state  transition  (discrete)  matrix. 
Sequencing  the  time  index  k we  obtain  for  Eq.  (2) 

X(l)  = $(1,  0)  X(0) 

X(2)  = $(2,  0)  X(l)  = $(2,  1)  t(l,  0)  X(0) 


X(k)  = <I>(k,  k-1)  X(k-l)  = $(k,  k-1)  .. 
X(k+1)  = $(k+l,  k)  X(k)  = $(k+l,  k)  .. 

or  packagewise 


$(1.  0)  X(0) 
$(1,  0)  X(0) 


or 


! x(i) 

r 

$(1,  0) 

X(2) 

$(2,  1)  $(1,  0) 

X(3) 

$(3.  2)  $(2,  1)  $(1,  0) 

X(k) 

$(k,  k-1) 

f(l,  0) 

X(k+1) 

— -J 

4>(k+l,  k) 

~X(1) 

"$(1.  0) 

X(2) 

. 

“ 

$(2,  0) 

X(0) 

X(k) 

• 

♦(k,  0) 

X (k+1) 

<Kk+l,  0) 

X(0) 
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The  last  equation  in  the  batch  of  Eq.  (3)  was  obtained  by  substituting  in 
from  the  top  down.  One  can  also  work  from  the  bottom  up,  thus 

.-1, 


X(k)  ■*  4>  (k+1,  k)  X(k+1) 

X(k-l) 


4>"^(k,  k-1)  X(k)  = «"^(k,  k-1)  <l>"^(k+l,  k)  X(k+1) 


(6) 


X(l) 


r^(2,  1)  X(2)  = *■^(2,  1)  r^(3,  2) 


X(0)  = ♦‘^(1.  0)  X(l)  = $"^(1,  0)  $‘^(2,  1) 


♦"^(k+l,  k)  X(k+1) 
4>"^(k+l,  k)  X(k+1) 


and  packagewise 


or 


x(0) 

I-  ^(1,  0)  4> 

X(1) 

4>~^(2,  1)  $“ 

X(2) 

■ 

• 

X(k) 

^"^(k+l,  k) 

r-1  1 

'x(o)' 

4>  (k+1,  0) 

X(l) 

4>"^(k+l,  1) 

X(2) 

$"^(k+l,  2) 

i . 

1 

I^X(k) 

4>"^(k+l,  k) 

^■^(k+l,  k) 


X(k+1) 


(7) 


X(k+1) 


(8) 


It  can  be  seen  by  Eq.  (5)  the  k+1  matrices  are  a function  of  the  initial  matrix. 
Likewise  when  one  replaces  the  matrices  with  j=l,  or  a single  vector,  then 
the  state  vector  at  the  k+1  time  points  is  a function  only  of  the  initial 
vector  x(0)^  . 

If  now  4>  is  a constant  for  all  k,  that  is 
4>(k+l,  k)  = 4>(1,  0)  = $ Vk 

then 


4>(k+l,  0)  = $ 


(k+1) 


(9) 


and 

4>"^(k+l,  0) 


(10) 


l_x(k+i)j 

If  we  define  the  matrix  of  Eq.  (12)  as 


1 

$ 

J - 


0j(k)$ 


and  seek  the  psuedo  inverse  of  Eq.  (13). 
The  matrix  $ is  always  full  rank,  hence 
0*(k+l,  0)  = 0*. 


The  powers  matrix 


Using  Eq.  (10)  in  Eq.  (8) 


X(k+1) 


Using  Eq.  (9)  in  Eq.  (5) 


(k+l)pxp 


0(k+l,  0)  = = 4> 


(17) 


has  psuedo  Inverse 

0*  - 


px(k+l)p 

where 


/*T  /T  *21  .kT. 

/♦  - (I,  fli  , ^ i ) 


(18) 


and  the  full  rank  Grammlan  is 

.1  + 4.’^  + 4>'^^  4>^  + ...  (19) 

whose  Inverse  is  needed 

( )~^  = [l  + 4'’^4>  + . . . + 4)'^*^  4>‘^]"^  . (20) 

Eq.  (20)  appears  horrendous,  however,  we  shall  "construct"  a psuedo  inverse. 

By  Eq.  (12) 


X(0)  - 4>~^  X(l) 
X(0)  - 4>~^  X(2) 


(21) 


or  packagewise 

x(0)  = I [*"^,  p(i)\ 

\X(2)/ 


(22) 


or 


X(0)  - j [l,  AdA 

\X(2)/ 


(23) 


or  if 


0(2,  0)  - /I\  « = 0j(2,  0)  $ 


() 


0*(2, 

0)  - 

n 

:i,  L^,  4 

L 

\4./ J 

m 

(I. 

also 

0*  (2, 

, 0) 

4(1. 

, «'^) 

(24) 


(25) 


(26) 


(27) 


532 


By  definition  of  the  psuedo  inverse 

[I. 


0*  (2,  0)  = 

(I,  4'^) 

U/ 

f [(I,  ♦■^)]  = (I  + ^ (I. 

Multiply  Eq.  (28)  by  (I,  and 

1. 

2 


J (I,  1 (I  + $' 


T -1 
t)  ^ 21 


or 


(I  + = f [l  + 


The  first  three  elements  of  Eq.  (12)  are 


X(0)  « 4>  ^ X(l)  = 4>"^  X(2)  = X(3) 


or 


X(0) 


4 (t  $'^)  / X(l)> 


( x(2) 
\x(3)> 


or 


x(0)  = i ^ [l,  $ $'^lx(0)N. 

(x(l)j 

\X(2V 


hence  by  Eq.  (12)  the  first  three  elements  are 

'X(l)'| 

|X(2)i  = 


I 


•t'  X(0)  = 0(3,  0)  X(0) 


and 


0*  (3,  0)  = i [l,  4-“^, 


also 


0j  (3,  0)  = 

•t-, 
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r"  » 


and 


0*  (3,  0)  - i (I, 


(37) 


for  by  Eq.  (36)  and  Eq.  (37) 


0*  (3,  0)  0j  = I 


(38) 


By  definition  of  0*  (3,  0) 

.2T.  ^ ;vl  .T  .2T.  1 r,  .-1  ^-21 

(I,  4>  , (|)  ) \ (I,  ♦ , * ) = [I  ^ $ J 

$ 

Multiply  Eq.  (39)  by  [l,  $ 'I'  ^ 


(39) 


and 


but 


[I  + + (-f^^)^]  ^ = -I  [l  + ($^$)"^  + 4>~^ 


,-2  ,-2T  ^ (,2T  ^2^-1 


hence 


[l  + + ($^$)^  ] ^ [l  + («'^4>)  ^ + (4'^^  <I>^) 


Define  the  Grammian 
T 

$ <1>  = G 


then 


[I  + G + G^]' 

3 


[l  + G~^  + g"^] 


Continuing  in  the  above  manner  one  can  show  that  (by  Eq.  (2)) 
0(k+l,  0)  = 0 (k,  0)  $ =ri  ' 4> 

■*■  i 1 

, ^ I 


; 


(40) 


(41) 


(42) 


(43) 


(44) 


(45) 


and 
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(46) 


0*(k+l,  0)  - <1>"^  0*(k,  0)  » [l,  ...  4-"*^] 

^ k+1 


Consider  next  Eq.  (1) 


X(0) 

X(1) 

X(k+1)  = 

♦-(k-1) 

1 

. 

. 1 

i 

J j 

~X(0) 

X(l) 

= 4'(k,  k+1)  X(k+1) 

(k+l)pxp 

•t"  X(k+1) 


X(k) 


.-1 


'I'jCk,  k+1)  $ " X(k+1) 


One  can  construct  the  psuedo-inverse  as  before,  for  example 


'XCOX  = ^ ^ X(2) 


O'C ) 


and 


X(2)  = j $[$,  I]  /X(0)\ 
\X(1)/ 


and  in  general  if 


(j> 


•Fj  (k,  k+1)  = 


-k 


4> 


-(k-1) 


f*(k,  k+1)  = ...  $,  II  1 

^ k+1 


and 


4'*(k,  k+1)  - 4'*(k,  k+1)  ♦ 
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(47) 


(48) 


(49) 


(50) 


(51) 


(52) 


(53) 


(54) 


Section  21  NON-HOMOGEN EOUS  DISCRETE  DYNAMICAL  PROCESS 


This  section  considers  the  non-homogeneous  dynamical  process 
x(k+l^  = 4>(k+l,  k)  X (k)^  + B(k)f(k)^ 


(1) 


with  4>  a pxp  matrix  and  B a pxg  matrix.  For  a sequence  of  j trajectories,  we 
have  a rectangular  matrix  relation 


X (k+1)  = 4>  (k+1,  h)  X (k)  + B (k)f(k^<^)  1 
pxj  pxp  pxj  pxg 

F(k)  = f(k)><;^l 

for  k = 0,  1,  2,  ...  and  j ^ p 

Chasing  the  time-points  (index  k)  as  before  we  obtain 
f X(l)  = <t(l,  0)  X(o)  + B(o)F(o) 

X(2)  = $(2,  1)  X(l)  + B(1)F(1) 

X(2)  = 4'(2,  1)  $(1,  O)X(o)  + B(o)F(o)  + B(1)F(1) 

X(2)  = $(2.  1)4>(1,  O)X(o)  + $(2,  l)B(o)F(o)  + B(1)F(1) 

X(2)  = $(2,  O)X(o)  + B(l),  4-(2,  l)B(o) 


(ni)x 

Vf(0)/ 


X(3)  = $(3,  O)X(o)  + B(2),  $(3,  2)B(1),  $(3,  l)B(o)  /F(2)\ 

(m)j 

\f(0)/ 


(2) 

(3) 


(4) 


X(k)  = 4>(k,  o)X(o)  + B(k-l),  4>(k,  k-l)B(k-2), 


4>(k,  o)B(o)  /F(k-1)> 


^F(O) 


l^X(k+l)  = <l>(k+l,  o)X(o)  + B(k),  4>(k+l,  k)B(k-l),  ...  <I>(k+l,  l)B(o)  / F(k)> 


■F(0)/ 


Packaging  the  above  equations 


X(1) 

X(2) 

X(3) 


fd,  0) 
«K2,  0) 
4>(3.  0) 


X(0) 


• 

• 

X(k) 

4>(k,  0) 

X(k+1) 

4>(k+l,  0) 

’o  .. . 

0 

B(0) 

0 . . . 

0 

B(l) 

4>(2,  l)B(O) 

0 . . . 

0 B(2) 

4>(3.  2)B(1) 

■*>(3,  l)B(O) 

• 

0 

B(k-l) 

• 

*(k,  l)B(O) 

_B(k) 

'J'Ck+l, 

k)B(k-l)... 

KCk+l,  l)B(O) 

F(k 

F(k 


F(l) 

F(0) 


Using  the  linear  convolution  matrix 


'f(0)‘' 

■ 

F(l) 

F(2) 

F(k) 

F(k) 


F(l) 

F(0) 


Equation  (7)  becomes 

«>(1,  0) 


X(l) 

X(2) 


X(k+1) 

+ 


-tCkHl,  0) 


X(0) 


B(0) 

<J>(2,  l)B(O) 

B(l) 

4>(3,  l)B(O) 

«'(3,  2)B(1)  B(2) 

4>(k+l,  l)B(O) 

B(k) 

F(k) 


<!•  X(0)  + 3r 

p(k+2)xp  pxj 


The  state  of  Eq.  (A)  can  be  written  In  convolved  form  as 

k-1 

X(k)  = fl-Ck,  O)X(O)  + Y.  &fl)B(B)F(B) 

B=0 

and  the  (k+l)*"^  state  as 

k 

X(k+1)  = $(k+l,  O)X(O)  + ♦(k+l,  B+1)B(B)F(B) 

B=0 

If  one  Is  at  the  k^^  stage  and  wants  to  advance  m steps  ahead 

k+m-1 

X(k+m)  = 4>(k+m,  k)X(k)  + Yi  ♦(k+tn,  B+1)B(B)F(B)  i 

B=k 

Equation  (9)  Is  the  discrete  analog  of  the  continuous  convolution  Integral. 
If  we  lump  the  product  terms 

B(k)F(k)  = G(k)  I 


G(k)  "I  rB(k) 

G(k-l)  B(k-l) 


[G(0) 


and  Eq.  (1)  Is  (In  open  form) 
X(k+m)  = ^(k-hn,  k)X(k) 


_F(0)J 


+ f$(k+m,  k+1),  <I>(k-Hn,  k+2). 


<I>(k+m,  k+m-1),  l]rG(k) 

G(k+1) 


G(k+iiri-l) 


Multiply  Eq.  (14)  by  <I'~'*'(k+m,  k)  and  obtain 


4.  (k+m,  k)X(k+m)  = X(k)  + 4>"^(k+m,  k)[4>(k+m,  k+1),  ...]  rG(k) 


G (k+m-1) 


Now 


♦ ^(k+iE,  k)'l>(k+m,  k+1)  ■ ♦(k,  k-hn)'>(k+m,  k+1) 

- tCk,  k+1)  - ♦"^(k+l,  k) 


(16) 


♦ ^(k+m,  k)'l>(k+in,  k+ra-l)  = ♦(k,  k+m)'l>(k+m,  k+m-l) 

« 4>(k,  k+ra-l)  = ♦ ^(k+ra-l,  k) 

One  can  also  view  the  first  equation  of  the  set  (16)  as 

♦ (k+in,  k+1)  = ♦(k+m,  k)4'(k,  k+1) 

hence 

4>  ^(k+ra,  k)4>(k+in,  k+1)  = "I*  ^(k+m,  k)<t>(k+m,  k)4>(k,  k+1) 


(17) 


(18) 


etc. 


Using  the  relations  of  Eq.  (16)  in  Eq.  (15) 
X(k)  = 4>“^(k+m,  V)X(k+m) 


(19) 


-[^"^(k+l,  k),  4'"^(k+2,  k)  ...  4>"^  (k+m-l,  k) , 4>"^(k+in,  k)]  TcCk) 


G (k+m-l) 


or  index-wise 


k+m-l 


X(k)  = $"  (k+m,  k)X(k+m)  - Z 4>"'^(6+l,  k)G($)  . 

6=k 

Observe  that  by  the  x(k^  term  of  Eq.  (4)  for  the  j = 1 or  vector  case  and 
for  B = pxl  a constant  column  vector  and  for  zero  initial  state,  one  has 
for  <I>  a constant  matrix. 


J 

(20) 


x(k)>=  [b>,  -tb^,  i'hy 


."-lb>] 


K-1 


(21) 
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and  if  Che  kxk  matrix  of  Eq.  (21)  is  full  rank,  one  can  invert  the  matrix  and 
solve  for  Che  sequence  of  scalar  Inputs. 

By  Eq.  (9)  of  the  previous  section  one  can  write  Eq.  (7)  as 
X(l)  1 4>~^(k+l,  0) 

X(2)  = ♦"^(k+1,  1)  X(k+1)  + 6r  (22) 

• « 

• • 

• • 

X(k)  4>"^(k+l,  k) 

X(k+1)  I 

where  for  4>  a constant  matrix 


G(0)  - X(l) 

G(l)  - (-♦, 

• 

G(k)  = (-$,  I)  /x(k)  \ 
\X(k+l)j 

since 

X(k+1>  = <>x(k)  + G(k) 


(26) 


(27) 


or 


G(0) 

■ I 0 

’x(l) 

G(l) 

I 0 

X(2) 

G(2) 

0 --t  I 0 

• • 

• 

• 

_G(1') 

0 0 -4>  I 

* 

X(k+l)j 

(28) 


I 

o 

o 

-1 

■ I 

0 

— 

$ 

I 0 

I 

7 

4> 

4>  I 

= 

0 

-4>  I 

• 

. 

. 

• 

. 

. 

• 

^k 

*k-l 

<!> 

«>  , 4>  I 

0 

0 . .. 

I 

(29) 


Multiply  Eq.  (22)  by  Eq.  (29)  solves  for  the  sequence  of  output  forces  and  is 
called  "deconvolution". 


\ 

\ 

\ 
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SECTION  22 

STOCHASTIC  DISCRETE  DYNAMICAL  SYSTEM 


This  section  considers  the  non-homogenous  dynamical  process  of  the 
previous  section  but  with  a stochastic  term  added,  that  is,  the  sequence  j 

x(k+l)J>j  = (})(k+l,k)x(ki>j  + B(k)f(k)^+  u(k)^^  (1) 

where  without  too  much  loss  of  generality 

(j)(k+l,  k)  = (|)  V-  k (2) 


and 

B(k)  = B 

For  the  package 

X(k+1)  = (})  X(k)  + B f(kj><^l 
pxj 

The  mean  is 

X(k+1)  l'(j^=  u(k+l)^ 
or 

u(k+l)}>  = (})  w(k)^+  B f(k^ 

and 

x(k)^.  = y(k)^+  x(k^  . 

or  for  the  package 

X(k)  = y(kj><^l  + X 
using  Eq  (6)  (for  k)  in  Eq  (7) 

X(k)  = X(k)  l*(jy  ^1  + X 
or 

X(k)  = X(k)  [I^  - 
or 

X(k)  = X(k) 


(3) 

+ U(k)  (4) 

pxj 

(5) 

(7) 

(8) 

(9) 

(10) 
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By  Eq  (10)  we  see  that  the  orthogonal  complement  projector  maps 
Eq  (4)  onto  the  "residuals"  that  is 


X(k+l)P  = (f)  X(k)  P + B f}^!  P + UP 


11  11 


<)  1 = <0 


hence 


X(k+1)  = X(k)  + U(k) 


which  is  the  recursive  dynamics  for  package  of  errors. 

Transposing  Eq  (13) 

x’^(k+l)  = x’^(k)  (J)’^  + U^(k)  (14) 

The  i nn er- Gramm i an  of  Eq  (13)  and  Eq  (14)  is 

X(k+l)X^(k+l)  = (})  X(k)  x'^(k)  (jl^ 

+ (|)  X(k)  U^(k)  + U(k)  x'^(k)  (|)^ 

+ U(k)  l/(k)  (15) 

Consider  next  the  "cross"  terms  of  Eq  (15);  by  Eq  (13)  and  Eq  (4)  of 
the  previous  section. 

X(k)  = ())^  X(o)  + [I,  ((),  (f)^,  . . . |u(o)  1 


i U(k-l) 


X(k)  U ^(k)  = . 


; U(o)  U ^(k) 
U(l)  U '^(k)  I 


U(k-l)  U (k)l 


where  it  has  been  assumed  that 


X(o)  U (k)  = 0 (18) 

that  is  the  initial  states  are  independent  of  the  process  noise. 


Using  the  countably  infinite  set  of  discrete  trajectories  concept, 
that  is 


lim 

j-X» 


X(k+1)  X (k+1)  fl 
pxj  jxp  j 


H P(k+1) 


Equation  (15)  can  now  be  written  as 

P(k+1)  = ^ P(k)  (j)'^  + Q(k,k) 

+ p3(o,k)  ^ 

!Q(l,k)  ; 


(19) 


+ [Q(o,k),  Q(l,k)  . 


I 

Q(k-l,k^ 

Q(k-l,kr]  fl 

'(t,T 


1 


Tk-li 


(20) 


If  the  process  noise  vectors  are  serially  uncorrelated,  that  is 

Q(i,k)  = Q(i,i)  6(i,k)  (21) 


then  Eq  920)  reduces  to  the  more  familiar  discrete  variance  dynamics 

P(k+1)  = (j)  P(k)  (|)’^  + Q(k)  (22) 

Observe  that  the  discrete  case  is  much  simpler  to  deal  with  than 
the  continuous  case  where  the  dirac  delta  function  under  the  integral 
sign  is  needed  as  in  Eq  (73)  sec  (38). 
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t 


By  Eq  (5)  of  the  previous  section  for  the  package 

X(l)  i (J  ; I ' 1 y 


i y(o) 


X(o)  + 


|X(k) 

also  by  Eq  (22)  of  the  previous  section 


fxd) 


X(k+1 


-(k+1) 


I 0 


X(k+1)+  |)  ^ I 


H)\.  . . 


I ■ U(k-l) 


Transpose  Eq  (23)  and  take  the  "outer  product" 


! xd). 

■ X(2)| 


[X  "^(1),  X '^(2),  ...  X '^(k)] 


_X(k^l 

= ({)’  X(o)  X '^(o)  . . . (j)^'^] 

; 


U(o)  U (o) 

ud)  u "^(o) 


U(k-l)  U(o) 


U(o)  U '^(k-1) 


. U(k-l)  U ^(k-1) 


at  the  variance  - Covariance  level 


P(l,l)  P(l,2)  . . . P(l,k) 
P(2,l) 


P(k,l) 


P(o,o)  (f)'^ 


(f)^  P(o,o)  (j)'^ 


P(o,o)  (f)'^ 


+ B 


Q(o,o) 

Q(l,o) 


PCk,k) 


P(o,o)  (j)^^  . . 


Q(o,l) 


P P(o,o)  ^ 


Tk 


(f)^  P(o,o) 


Q(o,k-l) 


(2 


Q(k-1,0)  Q(k-l,k-l)  j 

j 

The  last  term  of  Eq  (27)  is  quite  messy;  just  consider  the  diagonal 
matrix  case  for  Q(i,k)  (no  serial  correlation),  then  the  last  term  alone 
is 


B 


Q(o,o) 

0 


0 

Q(l,l) 


I 

I 


(21 


Q(k-l,k-l) 


Q(o,o)  Q(o,o)((l'^  Q(o,o)(jl^'^ 


(|)Q(o,o)  (})Q(o,o){j)^+Q(l,l) 

it)^Q(o,o) 


Q(o.o)^^X-^)  I 


. . ' “kk 

with  the  last  row  last  column  element  looking  like 

= ({)’""^Q(o,o)(j)^'"“^^  + (j)^’""^^Q(l,l)(j)^’""^^  + 

+ ...+({)  Q(k-2),k-2)(t)^  + Q(k-l,k-l),  (29) 

Observe  by  Eq  (28)  that  if  the  process  noise  is  serially  uncorrelated 
that  the  (j)  matrix  provides  serial  correlation  on  the  states.  Also  even  if 
the  process  noise  is  zero  at  all  time  points  k,  the  random  initial  condition 
variance  is  seen  by  Eq  (27)  to  provide  serial  correlation  on  all  P(i,k) 
terms-  Note  further  by  Eq  (25)  the  rank  of  the  matrix  of  P(i,k)  is  reduced 
when  Q(i,k)  are  all  zero  to  rank  equal  p. 


SECTION  23 

ADDITIVE  NOISY  MEASUREMENTS  OF  THE  DISCRETE  DYNAMICAL  PROCESS 

Many  real-world  measurement  processes  are  approximately  described  as 
the  known  measurement  function  plus  additive  noise;  that  is 

z(k)^j  = H(k)  x(k)^j  + V(k)^j  (1) 

mxp 

Clearly  for  many  physical  and  engineering  applications  the  measurements 
are  non-linear  functions  of  the  states,  and  the  mxp  matrix  H of  Eq  (1)  is 
a matrix  of  partials.  In  the  same  context  as  the  previous  section  assume 
that  H is  a constant  matrix  (this  assumption  reduces  carrying  many  integer- 
tags  and  greatly  simplifies  the  typing. 

The  sequence  of  trajectories  of  Eq  (1)  is 

Z(k)  = HX(k)  + V(k)  (2) 

mxj 

As  before  defining  each  measurement  error  as 

z(k)^.  - u^(k)>  = z(k);>j  (3) 

operate  on  Eq  (2)  by  in  j -space 

Z(k)  = H X(k)  + V(k)  (4) 

Transposing  Eq  (4) 

Z(k)^  = X^(k)  h'^  + V(k)^  (5) 

and  forming  the  iniier-Grammian 

Z(k)z’^(k)  = H X(k)  x'^(k) 

+ HX(k)V^(k)  + V(k)  X^(k) 

+ V(k)  VCk)”^  (6) 

If  the  state  errors  are  independent  of,  that  is  serially  uncorrelated 
with  the  measurement  noise,  then 

X(k)  V^(k)  = 0 (7) 

and  Eq  (22)  for  the  uncorrelated  case  of  the  previous  section  in  Eq  (6) 
yields 

Z(k)  z'^(k)  = H P(k)  + R(k,k)  (8) 
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Z(k)Z  (k)  = H(fi  P(k-l)({i  + HQ(k)H  +R(k) 


(9) 


When  the  process  noise  and  the  measurement  noise  are  correlated,  one 
obtains  by  Eq  (17)  of  the  previous  section 


X(k)V^(k)  = 


U(o)  V (k) 


U(l)  V'(k) 


U(k-l)  V^(k) 


and  Eq  (6)  becomes  at  the  variance  level 


i~(k)  = HP(k)H  + R(k) 

' *7 


t (o,k) 
^uv 


t (i,k)  ; 

^uv 


i (k-l,k) 

J 


+ [z^'^^(o,k),.  . . ^^'^^(k-i,k)]rr 


k(i<-i)' 


(10 


(ii: 


By  Eq  (4) 


24.  DISCRETE  KALMAN  FILTER  DERIVATION 


The  derivation  of  the  estimation  equations  about,  a single-state  variable 

process  x.(k)  satisfying 
J 

Xj{k+1)  - ♦(k+1,  k)  Xj(k)  ♦ f(k)  ♦ u^U)  (1) 

can  be  made  without  dralvlng  too  heavily  on  vector-space  theory.  The  multi- 
variable case  In  the  following  section  does  require  the  reader  to  be,  or  to 
become,  familiar  with  matrix- an filys Is  methods. 

Henceforth  we  shadl  assume  that  at  each  stage  k,  the  obseirvatlon  on  the 
process  state-variable  Xj(k]  is  contaminated  by  noise,  hence  the  states  Xj(k} 

cannot  be  known  for  einy  k Including  k « 1 because  the  observation  has 'additive 
noise  on  It 

*.(k)  ■ h(k)  X (k)  + V (k)  (2) 

j V (3 


Note  that  the  permutation  on  the  claisa  of  initial  conditions  has  been  dropped. 
We  assume  that  it  is  merely  one  value  of  the  unknown,  Xj(l). 

We  shall  develop  the  mathware  for  a computable  model  of  a system  of  equations 
which  will  compute  at  each  stage  the  best  estimate  (in  some  mathematical 
sense)  of  where  the  state  variable  *j(k)  is  when  we  know  only  Zj  (k),  the 

noisy  instrument  output. 


The  state  variable  x(k)  of  the  process  driven  by  the  random  unknown  variable 

u,(k)  would  be  known  at  each  stage  k if  the  sensor  were  noise- free,  that  is, 
J * 

if  v.(k)  were  identically  zero.  With  such  an  ideal  instrument  (or  high  pre- 
cision device)  there  would  be  no  doubt  as  to  the  trajectory  traversed  by 
x.(k).  In  fact  we  could  even  compute  the  values  of  the  random  force 


Uj(l),  Uj(2) 


Uj(k). 


However,  when  the  measurement  Zj  (k)  is  uncertain  due  to  v j(k),  we  cannot 

truly  know  the  states  x.(k)  since  we  do  not  know  the  instrument  noise  Vj(k), 

J 

hence  cannot  compute  the  process  noise  Uj(k). 


The  next  question  is  how  to  obtain  an  estimate  at  each  stage  k of  the  unknown 
state  variable  Xj(k).  The  estimator  will  use  the  known  dynamics  ♦(k+1,  /) 

of  the  process,  the  known  instilment  function  h(k)  and  other  things  as  needed 
in  the  development. 


T 


The  following  mathematical  developments  are  stage-vise  tutorial,  that  is  at 
stage  k > 1,  stage  k = 2,  and  then  finally  at  any  arbitrary  stage  k,  the 
procedures  are  developed  for  the  scalar  case  and  then  for  the  vector  case. 

Those  readers  already  familiar  with  the  recursive-relations  of  Kalman  may 
proceed  to  the  general  scalar  case  at  stage  k. in  section  II  or  the  general 
vector  case  at  stage  k in  section  HL 


Scalar  Case 
-Stage  k *»  1. 

At  stage  k a 1,  the  process  is  in  some  unknown  initial  state  x.(l),  nie 

el 

noisy  instrument  output  Zj  (l)  is  available  if  we  want  it.  Since  we  want 

to  obtain  an  error  signal  based  on  the  difference  between  the  observation 
and  some  guess  about  what  the  instrument  should  measure  then  suppose  we 
guess  or  estimate  what  the  initial  state  variable  is  (based  on  no  observations) 
and  call  this  (1,  0). 

V 

Since  we  know  the  instrument  function  h(k),  then  we  can  estimate ' that  the 
initial  Instrument  output  should  be 

ijd,  0)  » h(l)  i{j(l,  0).  . (3) 

We  can  now  compute  an  error  signal. 

(1,  0)  = Zj^(l)  - Zjd,  0)  (4) 

which  is  the  difference  between  the  first  instrument  reading  and  our 
estimate  of  what  it  should  read. 


We  have  now  estimated  what  the  initial  process  variable  and  instrument 
variable  should  be,  have  computed  an  error  equation  (4  ) and  may  now 
correct  our  estimate  of  the  initial-state  with  no  observation  and  obtain 
an  estimate  of  the  initial  value  of  the  state  variable  after  having  used 
or  received  the  first  measurement,  or 

aj  (1,  1)  *^(1,  0)  + w(l)^  (1,  0).  (5) 

We  now  seek  a w(l)  among  an  infinity  of  different  "corrector  weights"  or 
"feed  back  gains"  at  stage  1.  *We  want  the  w(l)  to  be  used  at  stage  one 
everytime  we  recycle  the  process  (or  experiment)  regardless  of  what  the 
random  vectors  u and  v are.  v(l)  should  be  a function  of  some  statisticeil 

measures  on  u and  v,  however 


BEST  AV."Lf.BLE  COPY 


Ibe  first  nols|r  observation  is 
»j  (1)  ■ h(l)  Xj(l)  ♦ 

Using  ( 3 ) and  ( 6 ) in  ( 4) 

Tj  (1,  0)  « h{l)[xj(l)  - Xj(l,  0)]  ♦ Vj(l). 

Using  (7)  in  (5) 

(1,  1)  ■ Xj(l,  0)  ♦ w(l)h(l)txj(l)  0)]  + w(l)vj(l) 

« [1  - w(l)h(l)]  Xj(l,  0)  + w(l)h{l)xj(l)  + w(l)  Vj(l). 
We  now  define  two  new  errors  as  differences  of  the  previous  variables. 

Xj(l)  - ^j(l.  0)  * 


Xj(l)  - Xj  (1,  1)  = Xj  (1,  1).  (10) 

The  variable  x.(l)  is  the  actual  (but  unknown)  process  state  variable  that 
exists  on  trajectory  J.  The  variable  3Cj  (l^l)  is  the  estimate  of  Xj(l), 

having  used  the  firdt  noisy  observation. 

Hence ^ (1,  l)  is  the  error  in  our  estimate  of  the  state  at  stage  1 based 
on  one  measurement. 

From  equation  (3)  we  see  that  if  our  initial  guess  x.(l,  0)  is  wrong,  then 
z.(l,  0)  will  slLso  be  wrong  emd  after  our  observation  z(l)  comes  in  and 
onr  COTiputed  error 'z'.(l,  0)  of  equation  (4)  is  large,  then  by  equation  (5) 

V 

we  will^ have  a correction  term  which  is  the  product  (w(l)Zj(l,  l))  of  a 
large  term  times  the  weight  value  w(l). 

Continuing  with  the  derivation  of  the  weights,  we  obtain  the  error  term  by 
equation  ( S)  in  equation  (11  ) as 

^ (1,  1)  = * Xj(l)-w(l)h(l)x^(l)  (11) 

- [(l-w(l)h(l)]  Xj(l,  0)  - w(l)vj(l). 


(1,  1)  ■ (l-w(l)h(l)](xj(l)  - Xj(l,  0)1  - w(l)vj(l). 


Using  equation  (9  ) in  equation  ( 12 ) 

(1,1)  “ [1  - w(l)h{l)]  ^(1,  0)  - w(l)  Vjd)*  (13) 

If  we  square  the  error  of  equation  (13) 

Uj’d.  1)  * [l-w(l)h(l)]^(l,  0)  - 2w(l)vj(l)[l-w(l)h(l)]Uj(l,  0)  (14) 
♦ w2(l)v/(l). 

If  ve  take  the  partial  derivative  of  equation  (14)  with  respect  to  w(l)  and 
equalkto  zero  ve  obtain 

J^(l,  1)  = 2[1  - v(l)h(l)]  (-h(l))  5j(l,  0)  (15) 

5r^)  - 2vi(l)[l  - w(l)h(l)]x.(l.  0) 


- 2vj(l)[l  - w(l)h(l)]Xj(l,  0) 
♦ 2w(l)  v^(l)  » 0 


Taking  the  expected  value  over  all  experiments  in  which  .1  is  varying  Ve 
obtain  _ 

(1,1)1  f o ■) 

^ li(l)  (16) 


w(l)  E * 0 


Define  the  variances 


Ejs^d,  o)j  =0^(1, 

E^vjd)]  =(T^d) 


= 0^.(1,  0)  = p(:.,  0) 


We  follow  a number  of  current  Kalman  papers  and  papers  about  Kalman 
Estimation  and  designate  the  variance  in  state  as  p(l,  0). 

The  cross  term  of  equation  (!’)  under  the  expectation  operator  is  zero 
when  the  following  conditions  hold. 


e£[1  - w(l)h(l)]  Vj(2)  Xj(l,  0)J 
= [1  - w(l)h(l)]  e[vj(1)  Xj(l,  0)j 


We  now  assxane  that  our  initial  guess  of  x(l  ,0)  Is  Independent  of  ^^(1)  as 
ranges  over  all  allowable  values , that  is 


jljd)  ?jd.  0)^  = 


(Aider  these  assumptions  equation  (16)  and  (18)  become 
0 • -.{1  - w(l)h(l)]h(l)  p(l,  0)  ♦ w(l)0J^(l) 


-h(l)p(l,  0)  ♦ w(l)h^(l)  p(l,  0)  ♦ 


v(l)[h  (1)  p(l,  0)  •*•^^(1)]  * h(J-)  Pd.  0)  (23) 

w(l)  ■ h(l)  p(l,  0)[h^(l)  p(l,  0)  (24) 

The  value  of  p(l,  0)  which  by  equation  (17)  Is 

p(l,  0)  = - Xjd.  0)]  ■]  » guess  (25) 

can  be  obtained  ^ guess  or  by  a "learning  process",  or  by  a>priorl 
knowledge  about  the  system. 

The  variance  of  the  estimate  of  state  at  stage  1 based  on  the  observation 
at  stage  one  can  also  be  obtained  by  equation  ( 14)  as 

E (1,  1)J  = [1  - w(l)h(l)]^ 

when  the  cross  terms  and  zero,  that  is 

E^v^(l)  ^j(l,  0)J  2w(l)[l  - w(l)h(l)]  * 0.  (27) 

The  variance  terms  of  equation  (26 ) will  be  denoted  as 


CJ^d.  1)  = E^x^  (1,  dJs  p (1,  1) 


E^v2(l)}  =ayy(l) 

The  p(l,  1)  designation  is  in  keeping  with  the  now  near  classical 
notation. 

Expanding  the  terms  of  equation  (26) 

p(l,  1)  = tl  - w(l)h(l)]^  p(l,  0)  + w^(l)(r^'l) 

« [1  - 2w(l)h(l)  + w^(l)h^(l))  p(l,  0)  4 w^(l) 

» pd,  0)  - 2w(l)h(l)  pd,  0) 

4 pd,  0)  +o;yd)]. 


J 


Consider  the  last  term  in  actuation  ( 30)  and  use  equation  (24)  for 
then 


w^(l)(h^(l)  p(l,  0)  ♦^^.1)] 


- h^(l)  p^(l,  0)[h^{l)  p(l,  0)  ♦0;;^(1)]“^  [h^(l)  p(l,  0) 

• h^(l)  p^d.  0)(h^(l)  p(l,  0)  ♦^^(1)]"^ 

- h(l)  p(l,  0)  ^h(l)  p(l,  0)  [h^(l)  p(l,  0) 

Replacing  the  bracket  term  of  equation  (31)  by  v(l)  In  equation  (24) 
ve  obtain 

w^(l)[h^(l)  p(l,  0)  - h(l)  P(l.  0)  w(l)  I 

Using  ( in  -equation  (30)  ve  obtain 

p(l,  1)  - p(l,  0)  -2w(l)  h(l)  p(l,  0)  + h(l)  pd.  0)  wd)  ( 

« pd,  0)  - w(i)  hd)  pd,  0). 

pd,  1)  » pd,  o)[i  - w(i)h(i)] 

also  using  equation  (24)  in  equation  (33)  we  can  write  p(l,  1)  as 


pd,  1)  = pd,  0)  - pd,  0)h(l)[h(l)p(l,  O)h(l)  +^(1)]  p(l,0)h(l)  (34) 

: " ' " ■ ■ ■ ' ■ 

Equation  '34^  scalar  variance  (one  dimensional  uncertainty  ellipsoid) 

of  tho  estimate  of  the  state  using  all  of  the  current  observations. 

We  can  now  predict  what  the  next  state  variable  value  should  be  by  pro- 
pogatlng  forward  with  the  known  dynamics  as 

^(2,  1)  »4>(2,1)  x(l,  1)  + f(l).  (35) 

Ihe  prediction  of  the  next  observation  is 

s(2,  1)  « h(2) '5(2,1).  (36) 

Th**  elliposoid  of  uncertainty  of  the  predicted  state  is 

p(2,  1)  » 1)  2 (37) 


wh^  j 


x(2,  1)  » x(2)  - x(2,  1) 


r 


Using  equation  (1)  and  equation  (35)  in  equation  (38) 

'5c(2,1)  -(()(2,  1)7(1,  1)  '♦  u(l)  ( 

x^(2.  1)  -^(2,  1)  x^(l,  1)^(2,  1)  ♦ 2^2,  1)  x(l,  1)  u(l)  ♦ u^(l) 
The  expected  value  over  all  experiments  of  equation  (40)  is 

e£x®(2,  l)j|-  p(2,  1)  -(^(2,  1)  p(l,  l)(j)(2..  1)  ♦O^^d)  ( 

where 


Cross  tern  statistical  independence  assumptions  are 

E^7(1,  1)  u(l^  « 0.  ( 

etc. 

In  conclusion,  at  stage  one,  ve  do: 

Given 

(T  1) 

vv 

guess  or  estimate 

p(l,  0)  and  ^(l,  O) 
compute 

^(1,  0)  ■ h(l)  x(l,  0)  ( 

Measure  and  compute  error 

7(1,  0)  ■ z(l)  -^(1,  0)  ( 

Compute  weight  v(l)  equation  i2k  ) 

w(l)  = h(l)  p(l,  0)[h(l)p(l,  O)h(l)  ♦ ^(1)]'^.  ( 

Update  or  correct  state-estimate 

^(1,  1)  - ^(1,  0)  ♦ w(l)  7(1,  0)  ( 

Compute  ellipsoid  of  uncertainty  of  state  estimation  ( 


p(l,  1)  ■ p(l,0)[l  - w(l)h(l)] 


Predict  or  propogate  one  stage  Into  future  with  knovn  dynamlcB  the  follovlng 
three  variables: 

er 

Predict  next  stage. 

^(2,  1)  » 4(2,  1)  5(1,  1)  + f(l)  (49) 

Predict  next  observation. 

5(2,  1)  * h(2)  5(2,  1)  (50) 

Predict  ellipsoid  of  uncertainty  in  state 

p(2,  1)  • ^(2,  1)  p(l,  1)  ♦(a,  1)  + Oyy(l).  (51) 

Wait  for  stage  k « 2 and  second  observation  to  come  in. 

Stage  k » 2. 

The  second  observation  is  taken  at  stage  2 and  is 

(2)  - h(2)  Xj  (2)  + Vj(2)  (52) 

« 

where,  as  before,  z (2)  is  known,  but  x (2)  and  v- (2)  are  unknown. 

J J J 

We  compute  the  error  signal 

'tj  (2,  1)  = Zj..(2)  -5j  (2,  1)  , (53) 

The  sub-script  J can  now  be  dropped  for  simplicity  of  notation,  where 
it  is  henceforth  understood  that  x(2,  1)  for  example  means  the  best 
estimate  of  the  true  .1th  tra.lectory  based  on  the  sequence  of  instrument 
noises<!^  . 

7 

Using  equation  (50)  and  equation  (52)  in  equation  (53) 

7(2,  1)  = h(2)[x(2)  -5(2,  1)]  + v(2)  (54) 

we  now  have  computed  the  error  in  the  observation,  hence  we  can  correct 
our  estimate  of  what  the  state  variable  should  be  based. on. the  use  of  the 
second  observation,  that  is 

5(^,  2)  = x(2,  1)  ♦ w(2)  7(2,  1).  (55) 

As  before,  we  now  seek  the  "feed-back”’ or  correction- weight  w(2)  at  stage 
2.  Before  deriving  the  w(2)  expression,  use  (54)  in  (55) 

x(2,  2)  = x(2,  1)  ♦ w(2)h(2)(x(2)  - x(2,  l)]  ♦ w(2)  v(2)  (56) 

5(2,  2)  = (1  - w(2)h(2)]  5(2,  1)  ♦ w(2)  h(2)  x(2)  ♦ w(2)  v(2).  (57) 
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From  the  error  in  state  estimation  at  stage  2 using  the  second  observation  : 

as  before 

‘x(2,2)  » x(2)  - x(2,  2)  -[l-w{2)  h(2))x(2,l)  ♦ w(2)  v(2)  (58) 

Multiply  equation  (38)  by  Itself  to  obtain  the  square  of  the  error  term  as 
'x^(2,  2)  » [1  - w(2)  h(2)]^  x^(2.  1)  (59) 

♦ 2w(2)  v(2)[l  - w(2)  h(2)]  x(2.  1) 

+ w^(2)  v^(2). 

Observe  that  the  error  and  the  s^quare  of  the  error  in  equation  ( 59)  is  a 
function  of  w(2). 

In  order  to  select  a v(2)  vhlch  vlll  minimize  the  square  of  the  error  ve 
t€dce  the  partial  derivative  of  equation  (59)  with  respect  to  w(2)  and 

equate  this  "gradient"  term  to  zero,  hence  1 

* -2h(2)[l  - w(2)  h(2))  x^(2,  1)  (60) 

. ♦ 2v(2)[l  - w(2)h(2))  x(2,  1) 

♦ 2w(2)  v(2)  h(2))  1r(2,  1)  I 


♦ 2w(2)  v^(2)  * 0 


nie  assumptions  of  equation  (6-^)  can  be  relaxed  but  then  one  needs  a 
deeper  knowledge  of  multi-linear  algebras,  matrix-packaging  and  partitioning, 
matrix  psuedo-inverse,  etc.  Hence  such  "highly-colored"  or  correlated 
noise  cases  will  not  be  discussed  in  this  paper.  The  majority  of  published 
papers  on  Kalman  theory  make  the  implied  Gaussian  assumptions  Implied  by 
equation  (63}  for  arbitrary  large  k.  Computer  storage  problems  for 
correlated  noise  for  large  k eure-also  a problem. 


Solving  equation  (60)  for  v(2) 

v(2)[h(2)  p(2,.l)  h(2)  o^,j2)]  - h(2)  p(2.  1) 


or 


(64) 


(65) 


The  second  weight  can  be  computed  since  h(2),  0^(2)  are  assumed  known  and 
p(2,  l)  was  computed  at  stage  1. 

We  can  now  derive  the  expression  for  p(2,  2),  using  equation  (65)  and 
taking  the  expected  value  over  all  experiments 

p(2,.2)  = E {5^(2,  2)}  = [1  - w(2)h(2)J^  p(2,  l)  ♦ w^(2)o^^(2), 

with  assumptions 

E {v(2)  'x(2,  1)}  * 0. 

Expanding  equation  (66) 

p(2,  2)  = [1  - 2w(2)  h(2)  + w^(2)  h^(2)]  p(2,  l)  w^(2)o^(2) 


-(2,  2)  * p(2,  1)  - 2w(2)  h(2)  p(2,  1) 

♦ w^(2)  [h(2)  p(2,  1)  h(2)  + o^^(2)] 

Consider  the  last  term  of  equation  (69)  using  equation  (65)  for  w(2) 
w^(2)  [h(2)  p(2,  1)  h(2)  + Oyy(2)] 

-h^(2)  p^(2,  l)[h^(2)  p(2,  1)  ♦ o^(2)T^[h^(2)p(2,l)  o^(2)] 

= h^(2)  p^(2,  1)  [h^(2)  p(2,  1)  ♦ 0^(2)]“^ 


(66) 

(67) 

(68) 
(69) 


(70) 


and  by  equation  for  v(2)  ve  obtain 

w^(2)[h(2)  p(2,  1)  h(2)  ♦0*yy(2)]  « h(2)  p(2,  1)  w(2) 
Using  equation  (71 ) in  equation  (69) 

p(2,2)  « p(2,  1)  - 2w(2)  h(2)  p(2,  l)  ♦ h(2)  p(2,  l)  w(2) 
or 


Ve  can  now  predict  at  stage  k ■ 3 the  state 
x(3,  2)  - ♦O,  2)  x(2,2)  + f(2) 

« 

and  the  observation 

z(3,  2)  = h(3)  x(3,  2) 
and  the  state-variance  term 
p(3,  2)  » E {x^(3,  2)}. 

By  equation  (l)  and  equation  (76) 

x(3)  * ♦(3,2)  x(20  + f(2)  + u(2) 

x(3,2)  - ♦O,  2)  5(2,  2)  + f(2) 

and  subtracting 

x(3)  - x(3,  2)  - x(3,  2)  » ^(3,  2)  1^(2,  2)  ♦ u(2) 
Squaring  equation  (si) 

^5(^(3,  2)  -^(3,  2) '5:^(2,  2)<J>(3,  2) 

♦ 2^(3,  2)  x'(2,  2)  u(2) 

♦ u^(2). 
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(71) 


(72) 


p(2,2)  « p(2,  1)  - p(2,l)  h(2)  w(2)  (73) 

p(2,2)  - p(2,  1)  [1  - w(2)  h(2)]  (74) 

Equation  (73)  can  also  be  written  as  by  equation  (65)  in  equation  (73) 
p(2,  2)  - p(2,  1)  - p(2,  1)  [h^(2)p(2,l)  +02)]"^  h(2)p(2,  l) 


(75) 

(76) 

(77) 

(78) 

(79) 

(80) 

(81) 

(82) 


Taking  the  expected  value  • 

E {S^{3,  2)}  - ♦(3,  2)  E(3^?(2,  2)}  ♦(3,  2)  + E {u^(2)> 

0*“ 

p(3.  2)  « ♦(3,  2)  p(2,2)  ♦(3,  2)  + ay^(2) 

Sunmarlzing  the  steps  at  st^e  k 2,  then  ve: 
measure  and  compute  error 

^(2,  1)  » z(2)  - z{2,  l) 
compute  weight  w{2)  by  equation  (65) 


(83) 

(84) 


(85) 


w(2)  * h(2)  p(2,  l)[h(2)  p(2,  l)  h(2)  +^(2)]“^  (86) 

compute  corrected  estimate  of  state 

x(2,  2)  = x(2,  1)  + w(2)  'zi2,  l)  (87) 

compute  variance  of  state  equation  (74) 

(88) 

(89) 

(90) 

(91) 

wait  for  next  stage  or  third  observation  to  arrive. 

Stage  k. 

The  derivations  of  the  equations  will  not  be  repeated  for  stage  k,  the 
relations  will  be  based  on  the  mathematical  process  of  reasoning  by  analogy. 
The  treatment  of  the  multi-variable  or  vector  case  will  derive  the  relations 
at  stage  k,  but  will  not  develop  the  stage-wise  logic  at  k ^ i and  k = 2. 

Ve  have  available  from  previous  stage  predictions 

x(k,  k-l) 

^(k,  k-l) 
p(k,  k-l) 


p(2,  2)  = p(2,  1)  - w(2)  h(2)  p(2,  1) 

Predict  (update)  state  via  dynamics  equation  (so) 
x(3,  2)  = *(3,  2)  x(2,  2)  + f(2) 

Predict  observation  at  next  stage 

^(3,  2)  = h(})  i(3,  2) 

Predict  next  stage  state-varismce 

p(3,  2)  = *(3,  2)  p(2,  2)  ^(3,  2)  + o^^(2) 
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•Dd  stored  1^)* 

Measure  and  compute  error 

t’Ck,  k-l)  » z(k)  - z(k,  k-l) 

Compute  v(k) 

w(k)  » h(k)  p(k,  k-l)[h^(k)  p(k,  k-l)  ♦ o^^(k)l”^ 

(loBipute  corrected  state  estimate  *' 

^(k,  k)  *^(k,  k-l)  + w(k)  ^(k,  k-l) 

Compute  variance 


p(k,  k)  * p(k,  k-l)  [l  - w(k)  h(k)] 

Predict  next  state 

x(k  + 1,  k)  = ♦(k  + 1,  k)  x(k,  k)  + f(k) 

Predict  next  observation 

^(k  + 1,  k)  = h(k+l)  ^(k+1,  k) 

Predict  variance  of  state 

p(k+l,  k)  » ♦(k+1,  k)  p(k,  k)  ♦(k+1,  k)  + o^^(k). 

define  the  noise  variances  In  the  notation  of  the  many  Kalman 
oriented  papers , that  Is 

* 

■ o;„(k)  - ,(k) 

• '■(k) 

The  three  familiar  equations  can  be  written  cm 
x(k+l,  k+1)  = ♦(k+1,  k)  x(k,  k) 

♦ p(k+l,  k)  h(k+l)(h(k+l)  p{k+l,  k)  h(k-H)  ♦ r(k)]"^ 
tz(k+l)  - h(k+l)  ♦(k+1,  k)  x(k.  k)l 
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(96) 

(97) 

(98) 

(99) 

(100) 

(101) 


p(k+l,  k)  a ♦(k+1,  k)  p(k,  k)  ((p(k+l,  k)  q(k)  ( 

p(k+l,  k+l)  a p(k-»-l,  k)  - p(k+l,  k)  h(k+l)  ( 

-I 

[h(k+l)  p(k+l,  k)  h(k+l)  + r(k)]h(k+l)  p(k+l,  k) 

We  shcdl  nov  define  In  words  the  meanings  at  stage  k of  the  variables  and 
rewrite  the  equations  using  the  distinguishing  J 


Xj(k+1)  »^(k+l,  k)  Xj(k)  + f(k)  ♦ Uj( 
Zjj(k)  a h{k)  Xj(k)  + 

x,j(k+l,  k+1)  =(}>(k+l,  k)  ^,.(k,  k) 


♦ p(k+l,k)h(k)[h^(k+l)  p(k+l)  + r(k)]"^ 

[Zj  (k+1)  - h(k+l)^(k+l,k)  (k,  k)]  (K 

p(k+l,  k)  *(|)(k+l,  k)  p(k,  k)(^(k+l,  k)  (K 

p(k+l,  k+1)  a p(k+l,  k)  - p(k+l,  k)  h{k+l)  (K 

[h^(k+l)  p(k+l,  k)  + r(k)]"^h(k+l)  p(k+l,  k)’ 

x(k)  a x.(k)  is  the  true  (unknown)  value  of  the  process  state  at 
stage  k as  a result  of  the  unknowns  u,(l),  ...  Vj  k) 
forcing  the- system.  ^ 

z.  (k)  a z,  (k)  is  measurement  of  the  true  noise  process  x.(k)  with 
^ additive  unknown  measurement  noise  v (k).  " 

(k,  k)  = x(k,  k)  is  the  best  estimate  of  the  state  at  stage  k of 
^ the  Jth  trajectory  based  on  past  observations 

up  to  stage  k,  that  is  recursively  we  have  used 
noisy 

Zj  (l),  Zj  (2)  ...  Zj  (k) 

made  noisy  by  v (l),  ...  u (k). 

A A, 

X.  (k+1,  k)  = x(k+l,  k)  is  the  best  estimate  of  the  state  of  the  Jth 
^ trajectory  at  stage  k+1,  based  on  observations 

only  up  to  k.  Also  interpreted  as  the  predic- 
tion of  the  state  at  next  stage  k+1,  based  on 
current  stage  k and  past  measurements. 
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VECTOR  ESTIMATION  EQUATIONS 


This  section  derives  the  Kalman  Estimation  Equations  for  the  multivariable 
case  using  matrix  analysis  methods.  The  derivation  techniques  are  the  same 
as  used  in  the  previous  scalar  case.  The  essential  difference  lies  in  the 
minimisation  methods.  The  variance  of  the  estimate  in  state  for  the  scalar 
case  is  a scedar  valued  function  of  a scalar  argument  v(k).  For  the  vector 
case,  the  trace  of  the  variance  matrix  of  the  estimate  of  state  is  like- 
vise  a scalar-valued  function,  but  a function  of  a matrix  of  p rows  and  m 
columns  Wvk).  The  minimization  of  a scalar-valued  function  with  respect 
to  thfs  matrix. 

One  can  eurrive  at  the  equations  via  strictly  algebraic  concepts  of  o *''ogonal 
projection  matrices  etc.,  in  which  one  does  not  have  to  enter  into  discussions 
of  partial  derivaties,  continuity  of  continous  variables  and  gradients. 

Since  the  majority  of  expected  readers  are  assummed  to  be  more  familiar  with 
the  least-squares  criterion  via  gradients,  this  report  will  stick  strictly 
with  this  method. 

The  general  linearized  vector  equations  are 

x(k+lH^=  §(k+l,  k)  x(kH^+  B(k)f(k)fe>  + N(k)u(k)^^  (l  ) 

/ ^ pxg  pxq  ^ 

z(k?^  = H(kJ  x(kj^^  + v(k)^  (2) 

The  deterministic  k-varying  vector  flk}^  is  of  dimension^g  less  them  or 
equal  to  p,  and  ^ts  distributed  cr  cro^-coupled  into  all  p state 
variables  x(k+l)f^  via  the  functional  relations  of  B(k)* 

^ nxg 


The  sane  statements  apply  to  the  noise  input  vector 


u(k)^. 


The  reader  should  keep  in  mind  the  families  of  tra.lectories  accurately 
descrii'ed  by  the  J and  r.  indices,  “bat  is 


x(k+l^^  = J(k+1,  k)  x(k)^ 
^ = H(k)  + v(^^ 


k)  x(k)(^  ♦ B(k)f(k^  + N(k) 


u(^ 


As  before^ the  accurate  descriptions  designated  by  J and  n will  be  dropped 
for  simplicity  of  representation. 

The  equations  are  developed  as  a "recursive  process"  or  an  "on-line" 
processor;  that  is,  as  the  odservat.ons  "role  in"  the  mechanized  computer- 
estimator  utilizes  the  data,  and  discards  it  or  stores  it  on  tape  or  what 
have  you.  All  past  data  is  sequentially  accumulated  in  the  "memory  of  the 


» . 


math-vare”  via  up  dated  estimates  and  variance  matrices  etc 


Suppose  ve  are  at  stage  k and  have  computed  during  stage  k-1,  the  following 


and  predicted  via  dynamics 


We  now  receive  the  kth  observation 


where  z(k^  is  known  but  x(k^  and  v(k^  are  unknown.  We  can  compute  an 
observation  error  vector  by'equation  \'i\  and  ( 9 ) as 


we  now  can  correct  the  estimate  in  the  state  vector  based  on  the  observable 
and  computable  estimate  in  the  observation  vector  as 


where  the  weighting  matrix  w{k)  at  stage  k has  p rows  and  m columns. 

We  next  seek  a procedure  for  selecting  a^,each  stage  a pxm  weighting 
matrix  W(k). 

Using  equation  (|0)  in  equation  ( ||  ) 

x(k,^^  = x(k,  k-^>+  W(k)  {H(k)[x(k^’ - 'x(k,  k-1^]  ♦ v(k^) 

= ( I - W(k)  H{k)]  x(k;  k-lj>  + W(k)H(k)x(k^  ♦ W{k)v(k 
pxp  pxm  mxp  ^ y 


(13) 


(14) 


If  we  now  define  the  "unknown"  error  vector* 

- x(k,  » x(k,  k^ 

and 

x{^^-  x(k,  k-^«^(k,  k-1^ 
then  theoretically  equation  (l2  ) in  ( 13)  yield* 

?(».  ^ - tl  - »(k)H(k)lx(i^-  ll  - W(k)H(k)J  i(k.  k->  (15) 

- W(k)v(k^ 

T(k.^=  [I  - W(k)H(k)]x(k,  k-^-  W(k)v{^  (16) 

Transposing  (16)  we  obtain 

<^k,  k)  «<[^k,  k-1)  [I  - H'’‘^{k)  w'^(k)]  - ^U)w'^{k)  (17) 

The  dyadic  product  of  equation  ( 16)  and  ^7  ) yields 

x(k,^^^^^(k,  k)  = {[I  - W(k)H(k)]  ^(k,  k-^^  - W(k)v(k^}  ‘ (18) 

(<^k,  k-1) [I  - H'^(k)  w'^(k)]  -<;^k)  w'^(k)} 

= [I  - W(k)H(k)]x(k,  k^l)X[^(k,  k-l)[l  - H’’(k)w'^{k)] 

- [I  - W(k)H(k)]^(k,  k-:^><^(k)w’’(k) 

- W(k)v(k^)<^k,  k-l)[l  - H'^(k)w'^(k)] 

♦ W(k)v{ki^X5^(^^w'^(^)- 

The  square  of  the  magnitude  of  the  error  vector  x(k,  kO  is  given  as  the 
inner-product  of  equation  ( 16  ) and  equation  (17)  or,  *£18  the  trace  of  the 
outer-product  of  equation  ( is)  as 

k)  x(k,^^^*^(k,  k-l)  ^k,  k-1^  - 2^(k,  k-l)W(k)Hx(k,k-r^ 

<x(k,k-l)H’'(k)W™(k)  ?(k.  k^  (19) 

- 2^(k,  k-l)W(k)  v(k^ 

♦ 2<^k,  k-l)H^(k)  W^(k)  W{k)  v(^ 

<^k)  w’’(k)  W(k)  v(l^ 
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Equation  ( 19)  is  a scalar  valued  function  of  a matrix  argument  W(k)  of 
size  pxm.  We  shall  take  the  partial  derivative  of  the  scalar  with  respect 
to  the  matrix  W(k), 


k)  x(k,^k^ 


term  by  term. 


By  equation  ( 19),  there  are  six  additive  terms,  each  term  will  be  handled 
via  the  "gradient"  methods  in  Appendix  B. 

The  first  term  is  not  a fxinction  of 


V(k),  hence 


a 

3W(k) 


^^k,  k-l)  'x{k,  k-^^  = 


[0] 


(20) 


The  second  term  is  by  equation 


a 

awTkT 


1^-2  <^k,  k- 


1)  W{k)  H(k)  x(k, 


-^U)x(k,k-l^^{k,k-l)  (21) 


The  third  tern  is 


_a 

aw 


{^(k,  k-l)  H'^(k)  w'^(k)  W(k)  H(k)  ^k,  k-1^} 
= 2H(k)  x(k,’k-l))x^(k,  k-l)  H'^(k)  w'^(k) 


(22) 


rhe  latter  derivation  is  based  on 


.)  H^(k) 


=<^(k,  k-l) 
b^  = H(k)  T(k,  k-1^ 
The  fourth  term  is  by  equation 


{-2^^k,  k-l)  W(k)  v(k'^  } * -2v{k^^(k,  k-l) 


The  fifth  term  by  equation  is 

— {2^{k,  k-l)  H'^(k)  w'^(k)  W(k)  r(k^ 


} 


(23) 

(24) 

(25) 

(26) 


{H(k)  x(k,  Il-l^^(k)  + v(k^^Tk,  k- 


i)h'^)  w"^ 
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based  on  setting 

in  equation  ( ) 

nie  sixth  term  is 

•5^  {<^(k)  W^{k)  W(k)  T(k^)  - 2T(k^^k)  w’’(k) 

Utilizing  the  above  six  expressions  in  equation  (19  ) after  the  partial 
derivative  has  been  taken 

k)  x(k,  k^J 

« -2H(k)  ^(k,  k-  ijX^k,  k-i) 

♦ 2H(k)^(k,  k-l^i^x1[k,  k-l)H'^(k)  W^(k) 

- 2v{k^>^xl:k,  k-1) 

+ {H(k)  7(k,  k-l!^><v(k)  + v(k^^k,  k-l)H'^(k)}  W^{k) 

♦ 2v{k)Vv(k)  w'^(k)  = [0] 

^ mxp. 

The  expected  value  over  all  experiments  and  allovable  values  of  J and  n 
yields 

k)  x(k  km  » -2H(k)  E{x(k,  k-  k,  k-1)} 


(27) 


(28) 


(29) 


♦2H(k)  E{x(k,  k-l^i^tk,  k-l)}  H'^(k)  W^(k) 

-2E{v(k^[^xtk,  k-l)} 

♦ lH(k)  E{^(k,  k-l)^(k)  v(k^^^^k.  k-1}  H'^(k)]  W^’Ck) 

♦ 2E{v(k^^(k)}  W^Ck)  » [0] 


(30) 


I 


If  ve  use  the  mtation 


P(k,  k)  «^jj(k,  k)  « E {xllk,  k)} 

(31) 

P(k,  k-1)  “ E{^(k,  k-^^^^k,'  k-1)} 

(32) 

Q(k)  « E{u(kr><^k)} 

(33) 

R(k)  * E {v(k^><^k)) 

(34) 

and  assume  that  the  conventional  statistical  independence  assumptions  bold, 


E{2(k,  k-lJ^Xtl(k)}  - [0] 

E{v(k^><^k-1)}  * [0] 

E{v(k^^k)}  » [0],  etc. 

By  equation  ( 31  ) through  ( 37  ) in  equation  (30) 


P(k.k)}  » -2H(k)  P(k,  k-1) 

3W^k ) . m m 

♦ 2H(k)  P{k,  k-1)  H (k)  W (k) 


♦ w’’(k)  = [0] 


Transposing 


W(k)[H{k)  P{k,  k-1)  H'^(k)  ♦ i(k)]  = P{k,  k-1)  H'^(k) 

* i>i« 


Inverting 


W(k)  « P(k,  k-1)  H'^(k)  [H(k)  P(k,  k-l)  H(k)  +T  (k)] 

” w 


k-1) 

(39) 

[■^(k) 

(40) 

(k)] 

r 

(41) 

(42) 

, k-1) 

(43) 

The  pxp  matrix  variance  of  the  estimate  of  state  (the  p-space  ellipsoid  of 
uncertainty)  can  be  obtained  by  taking  the  expeeteid  value  over  all  experiments 
of  the  dyadic  product  of  equation  (l8)« 

P(k,  k)  -E{(x'{k,  k^X^k,  k)}  (44) 

- [I  - W(k)  H(k)l  P(k,  k-l)[l  - H^(k)  w'^(k)l 

♦ W(k)  R(k)  w'^(k) 

Multiplying  out  the  terms  of  equation  (44  ) ve  obtain 

P(k,  k)  • P{k,  k-1)  - P{k,  k-1)  H^(k)  W^(k)  (45) 

- W(k)  H(k)  P(k,  k-1)  ♦ W(k)  H(k)  P(k,  k-l)  H^(k)  w’‘(k) 

♦ W(k)  R(k)  w'^(k) 

« P(k,  k-1)  - P(k,  k-1)  H'*^(k)  w'^{k) 

- W(k)  H(k)  P{k.  k-1) 

♦ W(k)[H(k)  P(k,  k-1)  H^{k)  ♦ R(k)]  W^(k) 

Consider  the  last  term  of  the  above  equation  and  equation  (42  ) for  W(k) 
with  the  transpose  ( 43) , then  the  last  term  becomes 

W(k)[H(k)  P(k,  k-1)  H'^(k)  + R(k)]  w'^(k)  (46) 

- P(k,  k-1)  H'^(k)[H(k)  P{k,  k-l)H^(k)  + R(k) r^[H(k)P(k,k-l)E’'(k)+R(k)  1 
x[H(k)  P(k,k-1)  H'^(k)  ♦ R(k)]"^  H(k)  P(k,  k-l) 

« P(k,k-1)  H'^(k)(H(k)  P(k,k-1)  ♦ B(k)]"^  H(k)  P(k,  k-l) 

- W(k)  H(k)  P(k,  k-1). 

Using  the  above  expression  for  the  last  term  in  equation  (45)  ve  obtain 

P(k,  k)  - P(k,k-1)  - P(k,.k-1)  H’^(k)  W^(k)  (47) 

- W(k)  H(k)  P(k,  k-1)  + W(k)  H(k)  P(k,  k-1) 
or 

P(k,k)  « P(k,  k-1)  - P(k,  k-1)  H’^(k)  w'^(k)  (48) 

P(k,k)  « P(k,  k-1) [I  - H'^(k)  w'^(k)]  (49) 

pxp  pxm  mxp 

The  matrix  P(k,k-l)  vas  predicted  and  computed  during  stage  k-1. 
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We  can  nov  predict  the  next  stage  (k'«-l)  state  vector  via  the  deteralnlstic 
pjrocess-dynami  cs 

x(k+l,^^  ♦(k+1,  k)  x(k,  B(k)  f(k^  . 

Die  next  stage  prediction  of- the  observation  vector  is 


c 


A 

X 


(k+1,  H(k+l)  x(k+l,  1^. 


it 


The  error  in  the  state  vector  at  stage  k+1  based 'on  the* prediction  of 
equation  ( SO  ) is 


A' 
x(k+l 


A, 

x{k+l 


x(k+^ 

I unknown  state  ve 
(k+^«=^(k+l,  + B(k)  f(^+  N(k)  u(^ 


it 


where  the  unknown  state  vector  is 


it 


and  the  unknown  error  is  by  equation  ( 50  ) and  equation  ( 53  ) in  equation 
( 52  ) 

♦ (k+1,  k)x(k)^  + B(k)  f(^ 


'^(k+1 


, k)^*  ♦(k+1,  k)x(k^ 

/ V X A N. 

+ N(k)  u(kj^  - ♦(k+1,  k)x(k,^  - B(k) 


it 


♦(k+1,  k)[x(k^  - x(k,  >] 

...  in  equation 

♦ (k+1,  k)  *x(k,  N(k)  u(k^ 


+ N(y)  u(^J> 


Using  equation 

x(k+l,^^^ 

The  transpose  of  equation  ( 55  ) is 

^^(k+1,  k)  =^^k,  k)  ♦'^(k+li  k)  +^(k)  N'^(k) 

The  dyadic  product  of  equation  (55  ) and  equation  (56)  is 
'x(k+l,  k+1,  k) 

■ ♦(k+1,  k)xl[k,  k^^(k,  k)^(k+l,  k) 

+ ♦(k+1^  k)  "^(k,  k^^^k)  N^(k) 


it 


it 


it 


+ N(k)  u(k^^^k,  k)^(k+l,  k) 
+ N(k)  u(kj^>^k)  N'^(k) 
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(58) 


The  expectation  ovei'  all  experiments  of  equation  (37  ) is 
E{x(k+1,  k^><^k+l,  k)}  « P(k+1,  k) 

- ♦{k+1,  k)  E{x(k,  k^^tk,  k))  •^(k+1,  k) 

♦ H{k)  E{u(k^)^U)}  N'^(k). 

The  statistical  independence  assumption  vas> inroked: 

E{x(k,  k^^k)}  ■ [0], 

Define  the  process  noise  variance  matrix  - ^ 

E{u(^^^(k)}»  Q(k) 
and  equation  becomes 

P(k*l.  k)  -^{k+l,  k)  P(k,  k)^’^(k+l,  k)  ♦ H(k)  Q(k)  H^.(k). 


(59) 


(60) 


Ve  may  nov  summarize  the  equations  and  the  computations 'to** be  performed  at 
the  kth  stage  as  the  kth  stage  summary. 


We  have  available  from  stage  k-i: 


x(k,  k-1^ 

z{k,  k-1^  - H(k)  x(k,  k-^  • 

P(k,  k-1).- 
Receive  z(l^^ 

Compute  error  in  observation 

^(k,  k-l^=  z(k^~  z(k,  k-1^ 
Compute  weight  matrix  W(k)  by  equation  (42) 


(61) 


-i 


W(k)  » P(k,  k-1)  H^(k)[H(k)  P{k»  k-l)  H (k)  + R(k)] 


(62) 


Correct  state  estimate  by  equation  (11) 


(k,  k^®^(k,  k-l^+  W(k)  z(k,  k-1^ 


(63) 
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(64) 


Conpute  new  state  variance  by  equation  (49) 

P(k.  k)  = P(k,  k-l)[I  - H'^(k)  W^{k)] 

Predict  to  stage  k+1,  by  equation  (50) 

x(k+l,  ^ = ♦(k+1,  k)  x(k,  B(k)’  f(l^ 


Predict  observation  by  equation  (51) 
^(k+l,^^=  H(k+l)  x(k+l,  k^ 
Predict  state  variance  by  equation  (60) 


P(k+1,  k)  = *(k+l,  k)  P(k,  k)  ♦’’(k+1,  k)  + N(k)  Q(k)  N'^(k) 


(65) 


(66) 


(67) 


wait  for  stage  k+1  and  new  measurement  vector. 

Hie  equations  can  be  substituted  and  Juggled  around  to  obtain  alternate 
e^ressionsjfor  exao-ile  xising  equation  (42)  and  (61)  in  (63)  we  obtain 

^(k,  x(k,  k-^^  (68) 

+ P(k,  k-1)  H’’(k)[H(k)  P(k,  k-l)  H'^{k)  + R(k)]"’- 

x{z(^-H(k)  .♦(k,  k-l)  x(k-l,  k-:^-H(k)B(k-l)f(k-^  } 


Many  similar  variations  of  the  above  systems  of  equations  occur  in  the 
literature. 

Stage  k = 1. 

The  vector  starting  system  of  equations  can  be  derived  from  equation 
Tor  k = 1, 

intelligent  guess 

H(l)  ^(1,^ 

and 

P(l,  0)  = intelligent  guess  based  on  experience  about  the  process. 
Receive  z( 

Compute  error 
'^(l,  ^ = 


(69) 

(70) 

(71) 


(72) 
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Conpute  first  velgbt 

tf(l)  « P(l,  0)  (H'^d)  P(l,  0)  H^(l)  ♦ R(l)]“^ 


Correct  state 


^ » xd,^^  ♦ Wd)  td. 

Coopute  state  variance  matrix  equation 


xd, 


PCI,  1)  - Pd,  0)  [I  :«  h’^Ci)  w’^d)]' 


Predict  stage  2 by  equation  (5®  ) 

IcCP,  (2,  1)  x{l,^^+  Bd)  fd^ 

Predict  stage  2 observation 
'z{2,  = H(2)  x(2,  ^ 

Predict  state  variance  matrix  bv  eouatlon  (67) 

p(2,  1)  = *(2,  1)  Pd,  1)  it(2,  1)  M(l)g(i)N'''(l) 


(73) 

(74) 

(75) 

(76) 

(77) 

(78) 
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EQUATION  SUMMARY  FOR  COMPUTER  APPLICATION 


This  section  summarizes  the  equations  of  the  previous  sections  and  points 
out  how  to  compute  mechanize  the  estimation  equations  to  recursively  estimate 
the  state  vector  as  the  observations  "roll  into  the  computer".  Precomputation 
of  the  sequence  of  weighting  matrices  for  large  dynamical  systems  is  a necessity. 

The  dynamical  process  is  described  by  the  state  vector  equation 


♦ (k+l,  k)X^>  + Bf(^  + Bfa>  + N(k)  U(^ 


(1) 


and  a system  of  noisy  instruments  whose  outputs  are  functionally  related  to 
the  states  by  the  observation  equation 


The  system  of  estimation  equations  are: 
The  state  vector  prediction  equation 


(2) 


A 

X(k+1 


4'(k+l,  k)X(k^+  B(k)f^^. 


The  observation  prediction 


Z(k+1 


,y> 


H(k+l)X(k+l 


> 


(3) 


(4) 


The  Observation  error 


and  the  correction  to  the  predicted  states  at  k+1  after  the  (k+1)^^ 
observation  is  available 


(5) 


X(k+1 


, k^ 


* 

X(k+1 


+ W(k+1) 


(6) 


The  sequence  of  weighting  matrices  W(k)  can  be  precomputed  and  stored  in 
memory.  The  weights  are: 


inhere 


W(k+1)  - P(k+1,  k)H(k+l)  |f^(k+l)P(k-H,  k)H  (k+1)  + R(k+lJ 


P(k+1,  k)  - ♦(k-H,k)P(k,k)'»^(k+l,  k)  + N(k+l)Q(k+l)N^(k+l)  . 


The  block  diagram  is  shown  in  Figure  (1)  as  the  conventional  feedback 
(discreet)  system. 


Z(k+1,  ig 


X(k+1,  ^ 


X{k+1,  10 


$(k+l,  k) 


FIGURE  (1)  - DISCREET  FEEDBACK  BLOCK 


By  re-arranging  the  positions  of  the  feedback  blocks  one  can  obtain  a 
flow-block  which  looks  more  familiar  to  a digital  programmer  as  shown  in 
Figure  (2). 

Tests  or  applications  in  which  one  can  plan  or  design  the  experiment  and 
the  times  k,  k+1,  etc.  at  which  instrument-data  will  be  used  to  estimate 
appear  to  admit  of  pre-computing  the  weights.  If  the  estimation  times  are  not 
pre-designed  one  must  compute  the  weights  on-line. 


5?(k+l.  S 


(k+1,  k) 
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W(k) 
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Figure  (3)  - Flow  Block  of  Estimator 


Section  25  WALSH  FUNCTIONS 


Walsh  Functions  are  highly  suitable  for  digital  devices  since  they  are  square 
waves  and  have  been  increasingly  successfully  applied  to  digital  communication 
and  two  dimensional  imaging  and  filtering.  Very  few  attempts  thus  far  have 
been  made  to  trajectory  estimation.  Dr.  C.  F.  Chen  published  the  first  paper 
to  my  knowledge  along  these  lines  on  October  1975  (reference  22)  in  a paper 
called  "Design  of  Piecewise  Constant  Gains  for  Optimal  Control  via  Walsh 
Functions".  It  is  hoped  that  these  techniques  will  soon  be  applied  to  optimal 
estimation . 

This  section  presents  some  of  the  known  and  published  properties  of  Walsh 
functions  and  adds  a few  new  relations.  Chen  also  in  reference  (20)  refers 
to  block  pulse  functions  very  closely  related  to  the  Walsh  Functions,  which 
he  says  in  theory  the  block  pulse  functions  do  not  form  a complete  set  while 
the  Walsh  Functions  do.  Chens  papers  in  reference  (21)  a excellent  state-space 
treatments  of  this  subject  matter. 

The  Rademacker  functions  (not  discussed  in  this  report)  are  basically  trigo- 
nometric functions  blocked  off  for  a fundamental  and  harmonics  as  shown  in 
Figure  (1) 


Figure  1 Rademecker  and  Sine  Function 


The  Walsh  function  (first  10)  are  shown  in  Figure  (2) 


The  first  f-^ur  block  pulse  functions  are  shown  in  Figure  (3) 


n 

Ln 


r 


The  block  pulse  functions  are  the  "building  blocks"  for  the  Walsh  functions 
and  the  relations  are  for  the  4x4  case 

■<fol  1111  Uq  ' 

(t>  11-1  -1  4« 

= ( 
<f2  1-11  -1 

<t>.  1-1-1  1 


^ 4x4 

or  for  the  m x m case 

(J)(t)(^  = W i|>(t)(i^ 

^ mvm 


The  inverse  of  the  Walsh  matrix  is 
W“^  = 1 W 


W^  = ml 
T 

W = W 


ii>(t^  = W (t)(t^ 


By  Figure  (2)  it  is  seen  that  continuous  derivatives  of  the  square  wave  functions 
do  not  exist  at  the  break  points,  however  integrations  behave  quite  well,  thus 
one  should  approximate  the  highest  derivative  of  the  function  and  integrate. 
Recall  that  with  the  polynomials  the  approximations  were  made  at  the  position 
level  and  velocities  and  accelerations  obtained  by  the  derivatives  of  the 
fitting  functions,  that  is 

<f  = <fV^  (8) 

Consider  approximating  the  derivative  of  a function  with  the  Walsh  functions 
x(t)  = <;m)(|)(t)  c(^  + x^(t) 

or  (9) 


x(t)  = I (t).(t)c 
i=0 


— : 

0(^  <^t)dt  = W I = W"^  (16) 

0 ^ m 


o 1 o K«f  17  f T '\ 


or  the  matrix  of  inner-products  commutes,  that  is 


= j (|)(t)><^il<(t)dt 


(18) 


0 

By  Eq.  (9)  we  can  approximate  the  function  in  block-pulse  function  base  as 
(errors  ommitted) 

x(t)  S<^(t)(t)$>  = <Ji)(t)c'^ 

or  by  Eq.  (16)  and  taking  inner-product 

0 

also 


( <l>(t^  x(t)dt  = (^  = W ^c*^ 
J n 


(20) 


(21) 


c**’.>  ^ WC> 

thus  a base  change  is  simple. 


(22) 


One  can  also  by  approximate  relations  rather  than  equivalent  relations  obtain 
the  monomial  base  in  the  Walsh  base,  that  is  if  for  example 


<f(t)  = (1,  t,  t^)  =<^t 


(23) 


then 

<t  = <*(t)C^j 

3x3 

that  is 

f (t)^  = <^(t)^^ 
i = 0,  1,  2 
Obviously 

f„(t)  = 0^(t)  = 1 

and 

t =<3»(t)^j 

or 


(24) 


(25) 


(26) 


(27) 
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By  Eq.  (24) 


etc . 

The  second  column  vector  has  approximate  values 


The  approximation  and  equal  signs  will  loosely  by  used  so  reader  beware 
For  example 


and 


(t),(t)tdt  = tdt  + I (-l)tdt  = -1/4 

■'o  ^ ■'o 


The  third  column  vector  of  Eq.  (28)  is 

^2  = 


1/3 

-1/3 

-1/8 


(35) 


J)t  = 4)Ht) 


(35) 

in  Eq. 

(24) 

1 

1/2 

1/3 

0 

1/4 

-1/3 

0 

-1/8 

-1/8 

(36) 


Note  that  if  one  approximates  the  first  three  monomial  base  functions  by 
large  number  of  Walsh  functions  say  m^3  then 


4)f(t)  = 4)4.c^^ 
mx3 


</(t)C*  = <J)())C 
3xm  ' 


(J>t^(})t 


The  inverse  of  the  matrix  is 


,-l 


>t 


-2 


8 


12  -4(8) 

-12  3(8) 


By  Eq.  (24) 


/ ^<tdt  = 
0 \ 


>t 


also 


; t^  <t  dt  - H . 
0 ^ 


11 


t^C^dt 


>t 


or 


"ill 


t^<^< 


dt 


where  H.,,  is  the  3x3  Hilbert  matrix, 
ill 


Transposing  Eq.  (24) 


T X 


<t>ty 


s£-?' "■ . f --w^: 


(37) 


(38) 


(39) 


(40) 


(41) 


(42) 


(43) 
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n 


hence 


> = 


Hill  S C" 

pt  pt 


and 


"ill  ~ V pt 

Since  the  Walsh  matrix  is  symmetric  by  Eq.  (1) 

< P =<PW 

and 

<^p  =<;  pW  1_ 

m 

If  we  partition  W into  its  column  vectors 
W = l^Q,  9^2’  ^3! 

then 

(PQ’  ‘'*2’  ‘>’3^  " K^O’  <B>1’<^‘*^2<^3^ 


(44) 


(45) 


(46) 


(47) 


(48) 


(49) 


or 


‘*’i  ^ ^'^i 


(50) 


and  the  . are  the  coordinates  in  the  block-pulse  base  of  each  of  the  Walsh 
functions . 


Likewise 


VV'°Av,' 

The  transition  matrix  on  the  column  vectors  of  Eq.  (51)  is 

^i+1  = ^d  '^i 
i = 0,  1,  2,  3 

where  the  permutation  matrix  or  shift-down  operator  is 


(51) 


(52) 


Sd  = 


'0  0 0 l\ 

10  0 0 


0 10  0 

,0010 


(53) 


'J 
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Walsh  function 


First  integrotion  of 
Walsh  functions 


Figure  4 

Walsh  Integration 

The  Walsh  functions  are  rectangular 

waves,  their 

first 

integrals  are  triangular  ; 

waves  and  the  second  integrals  are 

quadratic 

curves.  Chen  approximated  the 

triangular  waves  over  the  intervals 

as  shown 

in 

Figure 

(4) , to  obtain 

t 

1/2 

1/4 

1/8 

0 

/ <*(t)dt  =^(t) 

-1/4 

0 

0 

1/8 

(72) 

0 

-1/8 

0 

0 

0 

L ° 

-1/8 

0 

0 

J 

or 


/ ^(t)dt  = <^(t) 

where  the  integration  matrix  is  a constant. 


(73) 


The  8x8  integration  matrix  is  given  by  Chen  as 


P = 
w 

8x8 


1/2 

■1/4 

■1/8 

0 

■1/16 

0 

0 

0 


1/4 

0 

0 

■1/8 

0 

■1/16 

0 

0 


1/8 

0 

0 

0 

0 

0 

-1/16 

0 


0 

1/8 

0 

0 

0 

0 

0 

■1/16 


1/16 

0 

0 

0 

0 

0 

0 

0 


0 

1/16 

0 

0 

0 

0 

0 

0 


0 

0 

1/16 

0 

0 

0 

0 

0 


0 

0 

0 

1/16 

0 

0 

0 

0 


Chen  also  gives  the  16  x 16  matrix  which  is  the  same  as  a table  in  Corringtons 
paper  of  reference  (25). 
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with 


L 


4x4 


1/2 

1 

1 

1 


0 

1/2 

1 

1 


0 

0 

1/2 

1 


0 

0 

0 

1/2 


1/4 


(76) 


The  integration  matrices  and  P are  related  through  a base  change  (or  a 
similarity  transformation  as)  by  £q.  (46) 

<<t>  = <Jw 

and  integrating  both  sides  of  Eq.  77. 

=/<^4»dt  W 
or 

or  Eq.  (78)  in  Eq.  (79) 

<4.wp^  =<^PbW 


or 


P = w"^p^w 
w b 


One  can  now  easily  invert  the  Walsh  integration  matrix  for  by  Eq.  (81) 


p"^  = w"^p"^w  = wp'^w 

w b b 


(77) 


(78) 


(79) 


(80) 


(81) 


(82) 


where  it  is  know  that 


1 

-1 

0 

o“ 

-1 

’ 1 

0 

0 

0 

0 

1 

-1 

0 

1 

1 

0 

0 

0 

0 

1 

-1 

1 

1 

1 

0 

0 

0 

0 

1 

1 

1 

1 

1 

(84) 
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m 

The  block  pulse  matrix 

is 

lower  triangular  and  easily  invertable. 

i Also  note  that 

~~ 

n 

i / 

1 0 

0 

0 

^b  = ( 

1 1 

0 

0 

- 1/2  I ( 1/4 

(83) 

} 

1 1 

1 

0 

) 

K 

1 1 

1 

1 

r 

i 


where  the  shift-down  and  out  matrix  (shift  the  row  2 of  the  identify  matrix 
down)  is 


0 0 0 0 
10  0 0 
0 10  0 
0 0 10 


and  lower  triangular  unit  matrix  can  be  written  as 


1 0 0 

1 1 n 

1 1 1 

1 1 1 


2 3 

n 0 = 1 + s . + sj  + 

, . do  do  do 


One  can  write  the  inverse  of  Eq.  (84)  as 


(I-Sd„)-'  = (I-Sd/  l(I-Sd,)(I-Sd„)’'l'‘ 


The  symmetric  matrix 


= -J 


2-1  0 O' 

12-10 
0-1  2-1 
0 0-1  1, 


with  inverse  given  by  Eq.  (38)  see  (6)  as  the  Frankel  matrix,  that  is 


l»-®do>«-Sdo’’'l'‘ 


1111 
12  2 2 

12  3 3 

12  3 4 


and  Eq.  (89)  used  in  Eq.  (87)  yields  Eq.  (84).  One  can  now  use  the  Householder 
Inversion  Lemma  to  invert  Eq.  (83). 


One  can  also  use  the  relations  (not  derived  here). 


(I-S)'^  = I + S + 


(I+S)'^  = I - S + 


(I-S^)’^  = I + 


2 -1 

(I+S  ) = I 


and  for  a scalar  a 


-1  2233 

(I-aS)  = I + aS  + a S + a^S 


-1  2 3 3 

(I+as)  = I - aS  + a S - a S 


m 


S 


Note  the  class  of  matrices 


C = CqI  + c^s  + C^s 

D = d^I  + d^S  + d^S^ 

3 

commute , for  S =0  and 


(93) 


M 

(i,  S,  s^) 

/v\ 

CD  = (CqI,  Cjl,  c^I) 

s 

V 

i d.I 

i i 

1 2 / 

(94) 


= (CqI,  c^I,  c^I) 


^1 


d I 
\ 2 / 


CD  = Cgd^I  + (c^dg  + Cpd^)  S + (C^d^  + c^dj  + d^c^)S 

This  class  of  commuting  matrices  are  to  be  contrasted  with  the  commuting 
circulant  matrices 

C = aol  + a,P  a a/ 

with  the  permutation  matrix 

= I 

and  Eq.  (95)  becomes 


(95) 


(96) 


(97) 


(98) 


CD  (Cqj  ^2’  ^2*^ 


P 

I 


P‘ 

I 

P 


(99) 


Consider  a linear  constant  coefficient  differential  equation 
•x(t)  =/a  x^^  = a^x  + a^x  + a^x 

approximating  the  highest  derivative  with  Walsh  functions  or  block  pulse 
functions 

x(t)  =<^’‘(^3  - 'i>(t)c 

Integrating  (and  remembering  approximation  errors) 
x(t)  = x(to)  + <[^(t)Pj^^ 


(100) 


(101) 


(102) 


592 


x(to)  = ^^i|»(t^x(to) 


x(t)  = ^;^(t)(^jX(to)  + Pjj^l 


Integrating  again 


x(t)  = x(to)  + <;;[^(t)Pj^[^^x(to)Pj^^] 


= (t)  ='^;^(t)  [e^jX(to)  + Pj^^jX(to)  + P^c^] 


and  integrating  again 


x(t)  = x(to)  + <^4)(t)  [Pj^e^jX(to)  + P^^jX(to)  + P^c^l 
x(t)  =<^ilj(t)  jX(to)  + P^e^^x(to)  + P^  x(to)  + P^c 


Packaging 


♦ /i.c. 


L*  . 


terms 


hence  powers  of  the  integration  matrix  arise.  Note  that  power 


related  to  powers  of  P^  since  by  Eq.  (81) 


P = W P^W 
w b 


2 -12 

P = W P^W 
w b 


p*"  = w"^p"y 

w b 


From  the  foregoing  one  would  expect  to  exploit  the  cyclic  natu 
functions  just  as  has  been  done  with  Fast  Forrier  Transforms. 


Chen  presents  a method  for  solving  a non-horoogeneous  different 


^ = A ^ + f(t^ 


and  approximates  the  velocity  vector  with  say  block-pulse  func 


x^  = Cq)(t^ 


+ v^(t) 
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where  V (t)  is  the  approximating  error  and  will  be  ignored  in  the  derivations. 
Integrating  Eq.  (112) 


x(t)^  = x(to)^  * ^ 1 dt 

0 

(113) 

or  by  Eq.  (75) 

x(t)^  = x(to^  + CP^j4<(t)^ 

(114) 

Note 

x(to)^  1 = x(to)(^^) 

(115) 

since 

1 = <m)e  >l<(t^  i 

(116) 

hence 

x(t)'>  = [x(to^«\e+  CP  ] 4<(ti> 

Using  Eq  (117)  in  Eq  (112) 

i A[^Q<e  + CP^]  + ) ip>  = C.j> 

(117) 

(118) 

where  the  approximation  of  the  forcing  function  in  the 

block  pulse  base  is 

f(t)>  = c^q<(t)N 

(119) 

operator  Eq.  (118)  with  inner-product,  that  is 

f Cij/><Q|»dt  = C 1 
■'  0 m 

(119) 

hence 

A[x>o<e  + CPj^]  = C 

(120) 

or 

ACPb  - C = <e 

(121) 

Set 

-Ax>o<e  = B 

(122) 

then 

ACP.  - C = B 
b 

(123) 

One  can  solve  Eq.  123  for  the  rectangular  matrix  C via 
schemes  of  Eq.  (48)  Section  (G) . 

the  Kronker  matrix  product 

Using  Eq.  (124)  in  Eq.  (123) 


(P^  X A - I]  vec  C = vec  B 


(125) 


or 

vec  C=[P^xA-I]  ^ vec  B 
By  Eq.  (76)  and  Eq.  (2)  Section  (G)  for 


(126) 


P‘  X A = 1 
b 

m 


A/2  A 
0 A/2 


A 

A 


I 


0 


0 A/2 


(127) 


and 


P*  X A - I = 


I 


m2 

0 


A 1_ 
m 

A - I 
m2 

0 


A 1_ 
m 

A 1 
m 

A - I 
m2 


(128) 


Eq.  (127)  is  Block  upper-triangular  and  easy  to  invert.  Note  that  it  is  a 
generalization  of  the  matrix  of  F'j  '^'’3),  for  one  can  express 


p;  = (^i,  I,  I) 


uo 

2 

uo 


(129) 


where  S = S . 

uo  do 


and  the  (tensor  product)  Kronecker  matrix  product  becomes 


Pj  X A = (ijl,  I,  I)  11 


X A 


Thus  one  solves  for  the  matrix  C by  unpacking  or  de-vecing  Eq.  (126). 


(130) 


The  matrix  of  Eq.  (128)  is  a special  case  of  the  block  Toeplitz  mattrix. 
The  Toeplitz  matrix  is  of  the  form  for  a 3 x 3. 


T = 


-1 


-2 


•l 


-1 


(131) 
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I 


““  (3^2^ • ^2^^ 


(132) 


A matrix  closely  related  to  the  Teoplitz  matrix  is  the  Hankel  matrix  whose 
diagonals  run  from  top  right  to  bottom  left  or 


"-1  “-2  ' 

~ ^^2’  ^1’  ^0’  ^“1*  ^“2^ 


T2 

L S 
c uo 

T 

L S 
c uo 


L S 
c uo 

L 1 

c uo  ^ 


(133) 


(134) 


with  the  linear  convolution  matrix 


L = /O  0 1 

(010 


(135) 


1 0 0 


One  example  previously  encountered  is  the  block  bi-diagonal  matrix  and  its 
inverse 


I 

0 

0 a"^ 

1 

> 

1 

a-3 

> 

1 

A 

I 

0 =0 

A-l 

-A-2 

> 

1 

CO 

0 

A 

I 0 

0 

A-1 

-A-2 

0 

0 

0 

< 

0 

0 

A-1 

(136) 


Another  example  if  A and  B commute  and 


0 0 0 

A 0 0 

-B  A 0 

0 -B  A 


(137) 


then 


I 

I 


n-1 


a'^b 

(A’^B)^ 


0 

I 

a"^b 

(A‘^B)2 

.-1 


0 

0 

I 

a'^b 


,1 


(138) 


In  general  if  A and  B (and  A exists)  commute  and  the  special  Teoplitz  matrix  is 


T = 


then 


A 

B 

B 

B 


0 

A 

B 

B 


0 

0 

A 

B 


0 

0 

0 

A 


B B 


0 

A 


(139) 


n-1 


= A 


-1 


I 

C 

CE 

2 

CE 

CE" 


0 

I 

C 

CE 

CE*" 


CE 


m-2 


CE 


m-3 


(140) 


with 


C = a"^b 
E = I - a’^B 

It  is  seen  by  Eq.  (128)  that  the  block  pulse  integration  matrix  can  be  put 
into  the  transposed  form  of  Eq.  (139),  that  is 


P*  X A-I  = -1 
2m 


2mI-A 

0 

0 


-2A 

2mI-A 

0 


-2A 

-2A 

2mI-A 


(141) 


and  that  the  matrices  commute. 

Set  the  diagonal  matrix  of  Eq.  (141) 
L = 2mI-A 


(142) 
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and 


B = -2L“^A 


(143) 


then  Eq.  (141)  becomes 


P"  X A-I  - -L 
2m 


I 


B 

I 

0 


(144) 

Consider  the  inverse  of  the  unit-diagonal  upper-triangular  matrix  of  Eq.  (144) 


Multiply  out  the  elements  of  Kq.  (145)  to  solve  for 


A 

B 

-1 

I 

-B 

-B(I-B) 

0 

I 

B 

= 

0 

I 

-B 

V 

0 

V 

0 

0 

I 

(145) 


(146) 


and  using  Eq.  (146)  in  Eq. 

(144) 

”1 

-BL"^  -B(I-B)L"^  j 

[p][  X A-I]‘^  = (-2m) 

D 

0 

-bl"^  I 

(147) 

-1  i 

0 

0 L 

How  one  applies  Walsh  functions  to  budding  digital  filters  is  a wide  up 
untouched  area  as  far  as  trajectory  estimation  is  concerned.  Clearly  the 
cyclic-nature  of  the  functions  as  well  as  their  pulse  or  digital  device 
nature  excites  the  imagination  in  the  area  of  "real-time"  computations.  The 
cyclic  nature  of  the  trigonemetric  functions  have  served  continuous  analog 
devices  in  computing  and  communications  for  years;  perhaps  these  functions 
will  do  the  same  for  all  phases  of  trajectory  estimation  on  digital  computers. 
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SECTION  26 


I.  homogeneous  dynamical  system.  Consider  the  homogeneous  dynamical  sys- 
tem ~ 


where  A is  a constant  pxp  matrix. 

In  general  when  one  is  studying  the  effects  of  a linear  transforma- 
tion on  a vector  it  is  fruitful  to  study  the  effect  of  A on  a linearly 
independent  set  of  j^j  vectors  (that  is  any  basis  set);  for  example  if 

^ = Aj^  (2) 

where  x^  is  a p-dimensional  column  vector  then  for  a package  of  p 
vectors 

[y>i,y>2.---y>p]  =[ax>^,Ax>2,...a^p]  (3) 

or 

Y = A X (4) 

pxp  pxp  pxp 


If  the  background  base  vectors  have  coordinates 


0 

0 


Lth 


position 


then 


0 

1 

0 


0 

0 


0 


1 


(5) 


(6) 


and  if 


X = I 


(7) 
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then 


Y = AI 


(8) 


or  the  base  vector  I are  mapped  by  A to  vector  Y whose  coordinates  are 
the  columns  of  A. 

If  we  ask  what  linearly  independent  set  of  vectors  X are  mapped  by  A 
to  the  base  vectors  I,  that  is  find  the  X such  that 

Y = I (9) 

then 

I = AX  (10) 

or 

X = a"^  (11) 

for  full  rank  A. 

In  general  if  Y and  X of  Equation  (4)  are  given  then  we  can  solve 
for  A as 


A = YX"^  (12) 

The  inverse  of  a square  matrix  ( non- singular ) can  be  expressed  as 

X"^  = X^  (XX'^)"^  (13) 

Likewise,  if  we  have  an  over-specified  system  (that  is  more  equations 
than  unknowns)  or  more  than  a linearly  independent  set  of 

X^j^  i=l,2---p 


then 

Y = A X (14) 

pxj  pxp  pxj 

and  for  rank  of  X=p 


J 

.1 

.] 

I 


I 
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A = Y X* 
pxp  pxj  jxp 


(15) 


where  the  pseudo- inverse  can  be  written  as 

X*  = X^  (XX^)*  = (X^X)*  x'^  (16) 

jxp 


and  for  full  rank  case 


(XX^)*  = (XX^)"^ 


(17) 


Likewise  we  shall  consider  a package  or  sequence  of  p vectors  (or 
trajectories)  propogating  under  A of  Equation  (l)  such  that  at  time  zero 


X(0)  = 


or  each  trajectory  x(t^j  originates  at 
x(0^j  ; j=l,2,  • • ’p 


(18) 


The 

vectors 


dynamics  of  each  trajectory  or  the  velocity  of  each  of  the  p 
is 


i(t)^j  = A x(t)]]>j 


(19) 


and  the  dynamics  of  the  package  is  the  matrix  differential  equation 


X = AX 


(20) 


One  normally  assumes  the  solutions  to  (20)  to  be 

X(t)  = e^^  X(0)  = ■Kt)  X(0) 
or  the  solution  to  Equation  (19)  to  be 

x(t))>^  = e^^  x(0)]>j 


(21) 


(22) 
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and  takes  the  derivative 


X = A e^"*^  X(0) 


X = A X(t) 


where 


2 2 

A 4*  A "f" 

-Kt)  = e = I + At  + + • • • (25) 

By  inspection  of  Equation  (21)  if 

X(0)  = I (26) 

then  the  solution  of  Equation  (21)  is  a very  special  set  of  time  varying 
base  vectors  called  the  fundamental  basis  and  designated  as 


$(t)  = e"^  {21] 

The  time  varying  column  vector  of  $(t)  start  with  initial  condi- 
tions on  the  fixed  background  bases  and  propogate  under  A,  for  example 


4'(0)>^  = 


and  at  any  time  the  first  column  vector  has  velocity 


t>l  = A <t>(t)^ 


It  can  be  proven  but  will  not  be  shown  here  that  ♦(t)  remains  non- 
singular, that  is  the  vectors  in  the  column  space  of  ^ remain  linearly 
independent  over  the  finite  time  trajectory,  see  Equation  (25).  For 
example  consider  the  scalar  (one  dimensional  system) 


P 


eX^e  = CoH 


(34) 


Heace 


At  _ . , _ . A^t^ 

e - I + At  + — ;r-j — + — T-j — + 


or 


At  _ n- 


1 t 

LO  IJ 


^ = $(t) 


and 


x(t)  =1  1 f 
0 1 


x(0) 


(35) 


(36) 


Clearly  the  system  dynamic  matrix  A is  ra^  < one  (a  dyad)  by  Equation 
(33)  and  singular,  by  Equation  (35)  we  see  that  $ is  non-singular 
(invertible) . 

Since  the  columns  of  $ form  a base,  each  (t)(t^j  obeys  Equation  (1), 
or 


$ = A $ (37) 

where 

$(0)  = I (38) 

Fundamental  Solutions  and  Arbitrary  Initial  Condition  Solution  as 
Linear  Combination  of  Fundamental  Basis.  This  section  obtains  the  homo- 
geneous solution  for  an  arbitrary  initial  condition  as  a linear  combi- 
nation of  the  time-varying  fundamental  base  vectors  which  start  with 
initial  conditions  "on"  the  "fixed"  base  vectors  (non-time  varying  back- 
ground bases). 
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I 


i. 

r 

f 

r: 

i 

f. 


i 


Consider  a sequence  of  trajectories  generated  by  Equation  (19) 

= AX  (39) 

with  arbitrary  initial  conditions 

X(0)  = X (40) 

Since  't(t)  is  a set  of  base  vectors  (columns  form  a base)  the  jth 
vector  has  coordinates  in  the  $ base  as 

x(t))>.  = Ht)  (41) 

Taking  the  derivative  of  Equation  (41) 


(42) 


Using  Equation  (19)  and  Equation  (37)  in  Equation  (42) 

Ax^j  = A't'x‘*’(t)^j  + x***^.  (43) 

and  by  Equation  (41)  in  Equation  (43) 

A^x*^.  = A<tx*^.  + (44) 

or 

x‘*’>=  o)>  (45) 

which  implies  iQiat  x'f’^  is  a constant  vector;  that  is  the  coordinates  of 
the  vector  x(t^j  in  the  fixed  background  base  has  constant  coordinates 
in  the  fundamental  base;  furthermore  by  Equation  (41)  at  time  zero 

x(0)^  = lx'*’(0)^  (46) 

For  the  non-homogeneous  case  the  coordinates  will  be  shown  to  be 
time  varying  in  the  $ base  in  a later  section. 

Consider  next  the  dynamics  of  the  inverse  fundamental  system  of 
Equation  (37),  where 


or  taking  the  derivative 

. -1  .-1 
it  + it  =0 

or 

by  Equation  (37) 

i = At 


The  fundamental  inverse  can  be  written  as  two  ways 


Define  the  two  Graininian  matrices 


= G^.(t) 
91 


9 9 = G^Q(t) 

if  the  two  Grammians  are  equal  9 is  said  to  be  normal. 
Using  Equation  (58)  in  Equation  (57) 


9 = G, . 9 

9i 


Taking  the  time  derivative 


• • 'P  • »!» 

9 = G. . 9 + G. . 9" 

9i  <tii 


By  Equation  (55)  in  Equation  (61) 


A9  = G . 9"'^  + G .(-a'^9"'^) 


A99'^  = G^.  + G^.  (-a'^) 

<(ii  <(11 


and  solving  for  G^^ 


G = A G. . + G. . a' 

91  9i  9i 


which  is  a special  case  of  the  martix  Riccatti,  called  the  Lypunov 
relation. 
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The  dynamics  of  the  inverse  Grammian  G , . is  obtained  from 


or 


%i  = I 


d . G"^-  + G^.  G"^  = 0 
(^1  4)1  (^1  (^1 


’-1  -1  ■ -1 

G.T  = - G.T  G. . G. . 

(pi  (pi  (pi  (pi 


Using  Equation  (64)  in  Equation  (67) 


gT^  = - gT^  [AG.  ,-H3.  .a"^]  g7^ 

(jii  (j)i  (jii  (jii  (j)i 


or 


and  by  Equation  (64) 


G . = A G . + G , a"^ 

1 11 

By  Equation  (59)  the  time  derivative  is 

• • T T • 

G,  (t)  = $ 4>  + i $ 

(po 

and  by  Equation  (50) 

• T T T 

G,  = $ A $ + fl*  A* 

4)0 

• T T 

^ (A  +A)^ 

90 
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The  dynamics  of  is  obtained  from 

Ga  G"^  = I 
(^o  <tio 


• -1  •-! 

G.  G.  + G*  G.  = 0 
90  90  9o  90 


•-1  -1  • -1 

G/  = - G^  G^  G,^ 
90  90  90  90 


and  by  Equation  (59) 


-1  -1  -T 

G =9-^9 
90 


and  Equations  (73)  and  (77)  in  Equation  (76)  yields 

G"^  = - 9"^  (a'^+A)  9"'^ 

90 


a congruent  trans-formation. 

Summarizing  for  the  outer -Gramm ian  dynamics  by  Equation  (73)  and 
Equation  (78) 

G^  = (A+a')(^ 

90 

(79) 

G"^  = - 9"^  (A+a”^)  9~'^ 

90 

Returning  now  to  the  arbitrary  initial  conditions  for  the  homogeneous  case 
of  Equation  (20)  and  Equation  (21).  The  pseudo-inverse  or  dual  system  to 
Equation  (21)  is  given  by  Equation  (16)  where  we  have  two  Gram  matrices, 
again  one  p^p  in  size  the  other  j’<j  in  size,  or 


One  expression  of  Equation  (16)  is 


X*(XX'^)  = X'^(XX^)*(XX^)  = X^  (82) 

T T 

since  the  projector  (XX  )*(XX  ) acts  like  the  identity  for  the  vector  in 
the  row  space  of  X^,  hence  transposing  Equation  (82). 

X = (XX^)X*^  = G .X*"^  (83) 

XXI 

Taking  the  derivative  of  Equation  (83) 


• • ^ • T 

X = G .X*  + G .X* 

XXI  XXI 

Full  Rank  X(t).  The  solution  to  the  matrix  equation 
X = AX(t) 


(84) 


(85) 


is 


X(t)  = <Kt)  X(o) 


(86) 


Since  $(t)  is  full  rank,  the  rank  of  X(t)  is  determined  by  the  rank 
of  the  initial  condition  matrix  X(o).  If  X(o)  has  full  rank,  then  Equa- 
tion (86)  is  full  rank  factors  of  X(t),  hence  the  pseudo-inverse  is 

xjt)  = X*(o)  $"^(t)  (87) 

Taking  the  derivative  of  Equation  (87) 

X*(t)  = X*(o)  4'~^(t)  (88) 

By  Equation  (52)  in  Equation  (88) 

X*  = - X*(o)  (89) 


M 

' . i 

<14 

■ ' t 

- 

f 

« • 

and  by  Equation  (87)  in  Equation  (89) 

^ * 

ft  ■ 

X*  = - X*(t)A 

(90) 

or 

! 

ft  rp  rp  rp 

X*'  = - 

(91) 

I 

■ • 

The  dynamics  of  the  dual  (pseudo-inverse  transpose) 
for  the  full  rank  case  from  the  relation 

can  be  obtained 

1 

X X*  = I 

(92) 

pxj  jxp  pxp 

or 

XX*  + XX*  = 0 

(93) 

or 

: 1 

XX*  = - XX* 

(94) 

Multiply  Equation  (94)  on  left  by  X* 

X*XX*  = - X*XX*  (95) 


(99) 


X*^  = - A^X*^ 


Packaging  Equation  (18)  and  Equation  (99) 


X 

II 

A 0 “ 

x(tr 

x*^ 

T 

_0  -A  _ 

W)J 

(100) 


2pxj  2pxj 

The  inner-Grammian  by  Equation  (19)  in  Equation  (80)  is 
G^^(t)  = XX^  = $(t)  X(o)  xlo)  4>^(t) 


(101) 


or 


G^.(t)  = «t)  (102) 

The  dynamics  of  the  inner-Grammian  can  be  obtained  easily  from 

Gxxi^t)  = X(t)  x”^  + X(t)  x"^  (103) 


or 


G .(t)  = A G .(t)  + G .(t)  A^ 

XXI  XXI  XXI 


(104) 


For  the  full-rank  case 


G . G~^.  = I 

XXI  XXI 


(105) 


and  one  obtains 

G"^.  = [G'^.A+a'^G"^.]  (106) 

XXI  XXI  XXI 

Consider  next  the  commute  of  Equation  (92),  the  outer  projector 

P = ^ (X*X)  = X*X  + X*X  (107) 

xo  dt 
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1 


n 


By  Equation  (100) 


xo 


X*AX  + X*AX  = 0 


(108) 


Thus  the  time-derivative  of  the  time  varying  outer-projector  is  zero 
when  X is  full  rank  p.  Also  it  is  obvious  that  for  full  rank  p 


p ^ = 0 
xo 


(109) 


Homogeneous  System  Means  Averages.  The  dynamics  of  the  unweighted 
mean  of  the  j -trajectories  of  Equation  (39)  is 


where 


X 1*  = AXl*^ 

pxj 


l*(j>  = l(j>  J 

that  is  the  pseudo-inverse  of  the  sum  vector  ^ )1  where 
<j)  1 = (1,1,---1) 

ixj 

Define  the  mean  velocity  vector  as 
Xl*^  = ij^(t)(^ 

Xl’^=  Uj^(t)(p^ 


(110) 


and 


hence  Equation  (101)  becomes 

Partition  Equation  (114)  into  its  row  space  and  we  have 


(111) 


(112) 


(113) 


(114) 


(115) 
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(116) 


Xl*^  = 

II 

^<5)x  l*(p 

= 

p^(t) 

X 

• 

• 

^<j)x  1*> 

wP(t) 

or  in  its  column  space 

X1*>=  Qc(p^^--.x(p>.j 


j max 


I 

j=l  ^ 


j max 


(117) 


^(t)(p> 


Subtracting  the 
Equation  (19)  minus 

mean  vectors  from  each  of  the  j -trajectories. 
Equation  (115). 

that  is 

»>j  ■ ‘ 

^>3  =''(*>3  -“x>) 

(118) 

Define  the  error  vector 

^3  ' ^3  “ 

^x> 

(119) 

»>3  = *>j  - 

'^x> 

(120) 

or 

x^j  = Ax^j 

(121) 

or  package-wise 

X = AX 

(122) 

Consider  the  package  X by  Equation  (119) 
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I 


or 


P<x  = = P<(x  - P^xP^^ 

(128) 

or 

P<j)^  = ^4  '^^■^11^  = ^<^"^11 

(129) 

where  the  orthogonal  complement  projector  is 

P = I - P 

^11  11 

(130) 

Clearly  by  Equation  (114)  in  Equation  (124) 

X = X - x*:^4 

(131) 

X = xd-i’^^) 

(132) 

X = XP^^ 

(133) 

Thus  the  X and  X imply  orthogonal  and  orthogonal-complement  projectors 
respectively,  that  is 

X = X + X (134) 


where 

X = XP^^  = u^(^j)l  (135) 

is  a rank- one  dyad. 

Take  the  derivative  of  Equation  (135) 

X = XP^^  = AXP^^  = AX  (136) 

Thus  we  see  that  X,  X and  X all  have  the  same  velocity  matrix  A. 

By  Equation  (136) 

X = 

hence  Equations  (135)  and  (136)  in  Equation  (137)  yields 
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f 

f 


= * "x>4 


(138) 


The  dynamics  of  the  error  vectors  of  Equation  (131)  or  the  package 
(aggragate  or  ensemble  of  trajectories)  is  the  same  as  Equation  (136) 
except  for  the  wiggle  (-),  hence  the  dynamics  of  the  duals  of  the  errors, 
the  error  Graramians,  etc.,  are  the  same  except  for  notation  change,  hence 
we  have  via  Equation  (100) 


X = A O X 


X*'^  0 -A^ 


(139) 


If  we  define  the  average  inner-Graramian  as  a finite  variajice  matrix 


^ XX^  = ^ G.-  = I k- 

^m  ^m  j=l  ^m 

r\  J 


3™  IX 

'’m 


(140) 


The  dynamics  of  the  error  variance  is  given  by  Equation  (104)  as 


X 


.-1  -1  T.  -1 

I,  = - ^ A - A ^ 


Note  that  in  the  above 


(141) 


(142) 


d (^-1)  = j-1  ^ (£)-l 


(143) 


By  Equation  (133) 


X = XP, 


(144) 


we  see  that  the  rank  of  X is  j~l  since 
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p Pll  = j - 1 = p 


I -p 

jxj 


The  inner-Gramraian  is  pxp  and  for  the  rank  p case  we  have 
X, 


I-  = i 

pxp 


^^2 

■2pxj 


^■’'3 , r* 

3x2p  i + 


I 


r2i  f22_ 

and  in  partitioned  form  the  dynamics  of  Equation  (141)  is 


(145) 


(146) 


• 

i 

^11 

<12 

II 

J22 

• 

^22_ 

1 

1 

1 

^ll^ll'^^12^21  ^11^12'^\2^22 


^ll^ll'^^12^12 


11  21  ^12  22 


^21^11 '*'^22^12 


21"21  '>'22“22 


(147) 


Note  that  etc. , are  not  simple  matrix  Riccattl  equations  like  | 
of  Equation  (141). 


The  pseudo- inverse  of  Equation  (144)  can  be  written  as 


X*  (XX^)~^  = j x'^ 

m 

jxp  :,xp  pxp 


(148) 


The  inverse  can  be  obtained  by  partitioning  in  reference  (39)  as 

(149) 


'*11 

^12 

-1 

^ll(1.2) 

J2I 

^22_ 

X 

^2i^1’2) 

4(1*2) 
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where 


lll(l,2)  = ^^22"^21^11^12^  ^21  ^11 


^^^(1,2)  = - ^^2  ^^22~^21^11^12^ 


l2i(1.2)  = - 422“^21^11^12^  ^21  ^11 


(150) 


^22(1. 2)  = ^t22"^21^11^12^ 

If  we  partition  the  full  rank  pseudo-inverse  of  Equation  (148)  as 


X*  = 
jxp 


X*(l,2),  X*(l,2)~ 

_j  xn  j xn 


(151) 


where 


p = 2n 


XX*  =p^;  X*(l,2),  X*(l,2)^ 

pxp  ^ - -i 

= R^X*(1,2)  Xj^X*(l,2)'] 


X2X*(1,2)  X2X*(1,2) 


= P c- 

i 0 I 


(152) 


or  index-wise 


X;  X^<1,2)  = I 5.„ 


(153) 


Equation  (156)  can  also  be  written  in  terras  of  the  Graramians  via 
(155)  in  Equation  (156)  as 


xX  = 

s 

^1' 

-T  -1  ^ 

Xj,X2;  0 1 

1 ! -1  ! 

^2 

° '^11  ! 

or 


-1 

— 

«-l 

XX  = G 

G „ 

G_* 

0 

I 

G „G„r 

s 11 

12 

11 

12  22 

Si 

S2 

0 

S2 

=2i'=n 

I 

— 

— 

— 

Equation  (154)  can  be  written  in  temis  of  variances  as 


"2 


— - -tI  -T 
^ . ^1  ,-l 

^ra  ^ra 


X2X2 


-1 


-T 


-1 

22 


and  hence  the  package 


1 


L_ 


I 0 

I 


Using  Equation  (161)  in  Equation  (159)  we  obtain  the  product 
function  of  the  variances 


^11  ^12 

*u  0 

.s 

_^21  ^22_ 

0 

22_ 

Equation 


(159) 


(16C) 


(161) 


as  a 


or 


XX 

s 


(162) 


^ ^12^22 

♦21*U  “ 


The  off-diagonal  terms  are 

*1  *2  = 

nx  j 


and 


X2X1 


^ ^21^11 


(163) 


(164) 


can  be  related  to  correlation  matrices  in  very  simple  manner.  Consider 
the  2n  linearly  Independent  vectors"  in 

X 

2nxj 

or  the  2n  row  vectors  in  j -space,  we  can  say  for  the  first  n 

X^  e [X] 

nxj 

where  is  an  n-dimensional  sub-space  or  linear  manifold  of  j-space 

and  [ ] implies  spanned-by . 

Like-wise  for  X2.  Decompose  the  n vector  of  Xi  into  components  lying 
in  the  subspace  spanned  by  X2  and  components  out  of  X2  as 


Xi  = C^2^2  ^1  out  2 

and  like-wise 

X2  = + X2 


(165) 


(166) 
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ft  * 

Multiply  Equations  (165)  and  (166)  on  right  by  X2  and  respectively 


-ft 


X1X2  = C^2  + out  2 ^2 


^2"J  = Si  " S out  1 S 


(167) 

(168) 


Equation  (162)  can  be  written  via  Equations  (167)  and  (168)  as 


--ft  _ 

xx^  = ri  c 


12 


iSi  I 


1 out  2 2 


9 Jt 

2 out  11 


(169) 


If  the  n vectors  of  are  orthogonally  projected  onto  X2  as  shown 
in  Figure  [2] 


‘^j  )x^  nth  vector  of  X^ 


FIGURE  [2] 


ORTHOGONAL  PROJECTIONS  OF  ONTO 


then  for  the  package  of  vectors 


"1 

nxj 


^ '"I  out  2± 


(170) 


'1  [S2'^2j  ■ 

jxj 


and  likewise  for 


- r,*- 1 - 

X2  = X2  |_X  X^j  t X2  ^ 

jxj 

By  the  associative  property  for  matrix  products  one  obtains 
^1  ^ ^12j.  ^2  out  2± 

^2  ^ ^2U  ^1  ^2  out  U 

where 


1^1  = 

2U  ""  ^2^1 


(171) 


(172) 

(173) 


(174) 

(175) 


One  can  normalize  the  vectors  and  come-up  with  the  standard  correla- 
tion coefficients  as  direction  cosines  between  unit  vectors  or  as  angles 
between  sub-spaces  etc. , but  these  geometrical  concepts  will  not  be 
pursued  further  here. 

We  can  now  obtain  the  dynamics  of  the  singleton-duals  X*  by  Equation 


By  Equation  (139) 


F = xV 


(177) 


• • 

and  by  Equation  (147)  we  see  that  and  ^22  are  nastily  coupled  rela- 
tions with  the  needed  dynamics  f-1  being  difficult  to  come  by. 


by. 


The  transformation  between  the  singleton-duals  and  the  package-duals 
can  be  obtained  from  Equation  (161)  and  Equation  (148)  where 


= j^X*(l,2),xJ(l,2)J 

Using  Equation  (178)  in  Equation  (161) 
L^1’^2  ' ^ [Xi(1,2),X2(12) 


^11  ^ 


'■21'^11 


or 


* 

X = X 
s 


^12^22 


i 

^21^11 


which  yields 


* 

< 

s 


12 


^21  ^ 


22 


-1 
12’22 


* 

= XX 

^ ^12*22 

II 

_*  21*11  ^ 

_*21*U  ' 

(178) 


(179) 


(180) 


(181) 


which  agrees  with  Equation  (159). 


One  can  invert  the  matrix  of  Equation  (181)  to  obtain  the  inverse 
maps  of  Equation  (180) 


(193) 


T 


-1. 


= *(t)  B*  (t)  X^(t) 


(194) 


For  the  case  of  X having  rank  n,  the  Qy-t  2_  Equation  (170)  and 
Equation  (171)  are  equal  to  zero  and  we  see  by  Equation  (172)  and  Equa- 
tion (173)  that 


^12j.  ~ ^211  = 


(195) 


and  the  full  rank  factor  of  X ^6 


X = 
pxj 


"l“ 

I 

^2 

_T(t)_ 

X^(t) 


(196) 


The  dynamics  of  T or  T is  discuscad  in  the  next  section  via  Riccatti 
relations. 
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27.  PARTITIONED  HOMOGENEOUS  DYNAMICAL  SYSTEMS  RICCATTI  DIFFERENTIAL  EQUA 
TIONS  AND  ADJOINT  SYSTEMS.  Consider  a sequence  of  linear  coupled  homogen- 
eous differential  equations 


PlXPi 


P1XP2 


= ^^21  ^ 


A22  xCpX^j 


P2XP1 


P2XP2 


where  the  index  j generates  the  sequence  j>p  and 


P = Pi  + P2 


The  package  of  j trajectories  can  be  written  as 


"1  = ^ "12  ^1 


PfXj  A^^  A22  X2 


X = A X (4) 

Pxj  pxj 

In  this  section  many  relations  are  established  so  in  order  to  keep  the 
notation  simple  we  assume  pl=p2=n,  hence  p=2n. 

The  solution  of  Equation  (4)  is 

X(t)  = ({)(t)  X(o) 

2nxj  2nx2n  2nxj  (5) 

Since  (j)  has  rank  2n,  the  rank  of  X is  determined  by  the  rank  of  X(o). 
Suppose  X(o)  has  rank  n,  then  full  rank  factors  are 


X(o)  = B C 
2nxj  2nxn  nx] 
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Partition  (j)(t)  as 

<l)(t)  = 
2n  X 2n 


n X 2n 

n X 2n 


j 

( 5 ) can  be 

written 

as 

(t) 

= 

(t)^^  X (0) 

X (t)  = 

nxj 

nx2n  2nxj 

2nxj 

(t) 

X (0) 

nxj 

nx2n  2nxj 

Using  Equation  (6)  in  Equation  (8) 


where 


X (t) 


— 

(b,  B C 

= 

Y,  C 

^1  0 0 

1 0 

B C 

C 

“2  0 0 

2 0 

~ 

nxn  nxj 

k o 


k=l,  2 


nxn  nx2n  2nxn 

Equating  elements  of  Equation  (9)  by  proper  selection  of  X 
\ (t)  = (t) 


or 


or 


or 


^1  " ^o  = ^2 


^2  = ^2  ^1 


X^  = T (t)  X^ 
nxj  nxn  nxj 


where 


T (t)  = Y^ 


-1 


(7) 


(8) 


(9) 


(10) 

(11) 

(12) 

(13) 

(14) 

(15) 
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Note  also  that 


Y(t)  = ^ = (Jb 


o ' o 


Y = AY 


Using  Equation  (14)  in  Equation  (18) 


'1 ' 

2 U / "^3 


or  rank  n factors  which  is  the  same  as  Equation  (190)  in  the  previous 
section. 

Consider  the  dynamics  of  the  time  varying  transformation  between  the 
matrices  of  Equation  (14),  that  is 

= T(t)  Xj^(t)  (21) 

nxj 

If  X^  is  a full  rank  sequence  then  by  Equation  (21)  and  Equation  (15) 
^2  ^1*  = = ^2  ^l'^ 


X^  X^^(Xj^  = T 


Six  Six  = " 


where  the  inner-cross  Gramnian  is 


--'sr-jET? 
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or 


T --  -TA^^  * T - TA^jT  t A^^ 


(33) 


which  is  the  Matrix  Ricatti  Differential  Equation  for  the  system.  The 
initial  condition  for  Equation  (33)  is  given  by  Equation  (22),  it  is 
quadratic  and  non-homogeneous 


T (o)  = (o)  X^*  (o)  = Y2(o)  Y^"^(o) 


or 


T(o)  = X2(o)X^^(o)  G^^^(o) 


The  exponential-form  of  the  solution  of  Equation  (4)  is 


X(t)  = e^^X(o)  = (j)  X(o) 


where 


(|).(t)  = e 


At 


Note  that 


At 


and 


At  , 
e i 


11 


21 


• — 1 
-e- 

*^12 

l!" 

^22 

1 

— 

2 2 
• At 

+ . 

2! 

t 

"l2 

e 

t 

^^22’ 

e 

but  for  Block  diagonal  or  scalar  diagonal  systems 

o 


5“  ■> . 


^1^ 


i 


Using  Equations  (26)  and  (37)  in  Equation  (36) 


— . 

(t) 

I 

- 

X,  = 

X2  (t) 

T 

1 

mmm 

11 


^21 
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(34) 


(35) 


(36) 


(37) 


(38) 


(39) 


(40) 


(41) 


II 


i 


or  element-wise 


X^(t)  = + i)^2  T(o)]X3^(o)  (42) 

X^Ct)  = [^>23^  + P22  T(o)]X2(o)  (43) 


By  Equation  (21) 

X3^(o)  = T'^Vo)  X2(o)  (44) 

or  Equation  (44)  in  Equation  (43) 

X2(t)  = [(1)2^  t"^(o)  + 922^^2^°^ 

By  Equation  (22) 

T(t)  = X2(t)X3^’'(t)  (46) 

and  by  Equation  (42)  for  full-rank  factors 

X3^*(t)  = x’^(o)[(|)3^3^  + lj)j^2  T(o)]"^  (47) 

Using  Equations  (47)  and  (43)  in  Equation  (46) 

T(t)  = [921  ^^22  T(o)]C9ii  '^’12  (*^8) 

where,  by  Equation  (34) 

T(o)  = X2(o)X3^'^(o)CX3^(o)  X^^o)]"^  (49) 


Clearly  Equation  (48)  is  a solution  to  the  Ricatti  Equation  (33)  in  terms 
of  the  block  elements  ())ij  of  (p. 

By  Equation  (22) 

T(t)  = X2  X^^*  = (50) 

nxn 

Using  the  following  Grammians  and  cross-Grammians ; 


X 

X 

X 

X 


2 

1 

1 

2 


^x21i 

^xl2i 

®xlli 

^x22i 


jl) 


■ 


►KT  ^ ^ 0 •?^7j 


jH 
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'^2  - «yl2i 

- Vli 

^2  ^2'  = ^y22i 


By  Equation  (24) 


“ °x21^xll  “ ^y21  °yll 


By  Equation  (21) 


X2  = T(t) 


and  by  Equation  (15) 


Y = TY 
2 1 


Multiply  Equation  (53)  on  left  by  X^ 


or 


or 


> * 
X^Xj  = I = TX^X^ 


^ = ™lX2V22  - "<=.12^22 


^ ' °X22=X12 


or  by  Equations  (57)  and  (52) 

^ " ®x2l'^xll  ■ ®x22®xl2 
and  similar  analysis  for  the  Y’s  or 

' S2AU  • ^22  ^12 


(52) 

(53) 

(54) 

(55) 

(56) 

(57) 

(58) 

(59) 


One  can  now  obtain  the  dynamics  of  T in  terms  of  the  dynamics  of  the 
Grammian  and  Cross-Grammian  matrices  of  Equations  (58)  and  (59)  to  arrive 
at  the  results  of  Equation  (33);  thus  we  would  like  to  have  the  dynamics 
of  the  Grammians,  Cross-Grammians  and  their  inverses  for  the  X and  Y 
systems.  The  derivative  of  Equation  (59)  is 


T 


‘^y21  ®yll  °y21  ®yll 


(60) 
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By  Equations  (33)  and  (59)  equated  to  Equation  (60) 


T = -TA^^  + - T A^2 


-1 


°y21  °yll  ^y21  °yll 


■ "^y21  ^yll  ^11  * ^22  %2  %11 


'%21  ^yll  ^12  ^y21  ®yll  ^21 


Multiplying  Equation  (61)  on  right  by  G 


yll 


^y21  ■ “'^y21  ®yll  ^yll  ^ '^y21  ®11 


-1 


^22  ^y21  “ ^y21^®yll  ^12^^y21 


" '^21  °yll 


where 


11  yll  11  yll 


®22  ~ ^22 


®12  = ^yll  ^12 


®21  ^21  ^yll 


or 


>1 


^y21  ' " ^y21  ‘^yll  ^yll  '^y21  ®11 


®22  ®y21  " ^y21  ®12  ®y21 


+ B 


21 


The  special  cases  for 


G = 0 
yll 


or 


=yl2  = » 


will  be  discussed  in  later  sections. 


n 


By  Equation  (4)  we  have 
(^  = A (}) 


where 


({Xo)  = I = 
2nx2n 


Lo  ) 


Partition  (|)  as 


(|)(t)  = [(j)j^(t), 

2nx2n  2nxn  2nxn 


Form  the  inverse  in  terms  of  the  partioned  factors 

(f)"^  = (f)'^  ((fxj)^)”^ 

The  transpose  of  Equation  (67)  is 

Vi’ 


= 


and  the  Gramm ian  is 


♦2 

♦2*1 


♦1*2 

♦ 


2 ^2 


or 


= 

21  22 
Using  Equation  (71)  in  Equation  (68) 


^11 


12 


= G . 
1 


-1 


k-1 


(1,2) 

nx2n 

♦2  =1 

^2  (1»2) 

' 

nx2n 

(67) 

(1.^ 

I = 

^2) 

^2 
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W— 


(65) 


(66) 


(67) 


(68) 


(69) 


(70) 


(71) 


(72) 


(73) 


(|)^  (()^(1,2)  + (1,2) 

2nx2n  2nx2n 


By  Equation  (74) 


(jlj  <1)2(1, 2)  = I - <1)^  (1>^  (1,2) 
2nx2n 


where 

Vl  ' *1 

2nx2n 

is  a rank  n projector  since  by  Equation  (76) 

(t)^*(l,2)  = I 

nxn 


2nx2n 


By  Equation  (79)  we  see  that  the  projectors  are  orthogonal  complements 


Consider  next  the  singleton  Duals,  that  is 


♦2  ■ »2>''  ^2  ' %2«  <>2 


or  package-wise 


<i>2)  = (C’  F 


The  matrix  product  of  Equations  (87)  and  (67) 

. [♦fl  ((>1.  tj)  = f^u  0 g“ 

(t)  (b  = _i 

^ (j)*  ° 

-r  =u  •'iH 


S2  =21  I 


The  connection  matrix  between  the  two  pseudo  matrices  is 
6 = M (()" 


<^3  ^)  = M 

which  is  Equation  (88),  hence 


11  ^12  I 


^22  Si 


^1*2) 


and  by  Equation  (180),  section  (1)  the  inverse  matrix  relation  gives 


r 


(92) 


— " 

—1 

'tl*(l,2) 

‘ ♦ =12  =22  =21  =u 

■=12  =22 

<1)2(1. 2) 

-=21  =U 

' - =21  =U  =12  =22 

♦‘2 

1—  — 

The  connection  between  the  two  singleton  projectors  with  the  projectors 
of  Equations  (80)  and  (81)  can  be  established  by  Equation  (91) 

Ml  = M ^ hi  ^12 

M = h ^ h ^^12  '*’^^’2) 

M " Pii(l’2)  + <)*  G^2  (93) 


and  similarly  for  ^22’  anti-analogy  with  Equation  (74), 


^11  ^22  ^ ^ 


(94) 


Block  Orthogonal  Subsy terns  of  0,  Consider  special  case  where  the 

)ei 

>2' 


block  of  vectors  in  (|)^  are  perpendicular  to  all  of  those  vectors  in  block 
(^2  (on  subspace  spanned  by  ^^)  that  is  by  Equation  (71)  the  full-rank 
Grammian  is 


2 ^2 


or 


=12  = *2  = “ 


(95) 


(96) 


nxn 


Using  Equation  (96)  in  Equation  (93)  we  see  that  the  projectors  are 
equivalent , that  is 


and 


’ll  ” ^11(1.2) 

(97) 

'22  " ^22^^’^^ 

(98) 

’ll  ^22  ' ^ 

(99) 

put  the  additional  constraint  that 

(100) 
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then  Equation  (86)  becomes 


I 


f I 

f * 


I 0 

♦I 

0 I 

The  dynamics  of  these  two  systems  are  derived  in  later  sections. 


(101) 


Consider  next  the  sequence  of  vectors  contained  in  the  subspace  of 
(Jj^(t),  that  is 


x(o)^J^  = ().(o)  X (o)'^^ 

<t>lj 


2nxn 

and  package-wise 

t)l(o)  X^^(o) 

2nxj 

2nxn  nxj 

or  full-rank  factors. 

or 

to 

— « 

I 

X,,(o) 

X,,(o) 

X(o)  = 

0 

(J)l 

0 

2nxj 

L J 

as  a time  function 


2nxj 

or  by  Equation  (104) 


2nxj  2nxn  nxj 


or  full-rank  factors.  If  we  now  partition 


*♦!<»>  = 


X^(t) 

nxj 


X2(t) 

nxj 


(102) 


(103) 


(104) 


(105) 


(106) 

(107) 


(108) 


641 


and 


The  connection  by  Equation  (21)  is 


(109) 


(110) 


where 


V‘°’ 


(111) 


By  Equation  (109) 

0 = T^^(o)I 


^l'»>  = ° 

The  matrix  Ricatti  by  Equation  (33)  is 

"^(fl  ■ ■'^(|.l  '^ll  ^22  %1  ""^(fil  ^12  "^(1)1  ^^21 


(112) 


(113) 


(114) 


with  initial  condition  given  by  Equation  (113).  By  Equation  (48)  we  also 
have 

T^l(t)  = <l)2i(t)  (|)^];(t)  (115) 

which  is  the  solution  to  Equation  (114)  for  the  conditions  of  Equation 
(113). 


Consider  next  a sequence  of  vectors  belonging  to  the  (j)^  subspace,  that 


X(^  = (ll  (o)  X(o) 


(116) 


or  for  the  system  initial  states 


= ♦2 


(o)X.-(o)  = 0 

4-2  j 


S2(°> 


(117) 


(118) 


and  by  Equation  (118)  in  Equation  (117) 

X^^(t)  = (t)(t)|olx,,,(o) 


2(t)2' 


If  we  apply  Equation  (53)  to  Equation  (117)  we  obtain 

nxj 


or 


I = T(o)  0 

an  impossible  condition.  In  this  case  we  can  do  the  following 

for  now  we  have 

^l(t>2^°^  " ^24,2^°^ 

or 


0 = T^,(o)  I 


if 


If 


~ “ 

— 

— 

_ . 

X 

A 

A 

X 

1412 

_ 

11 

12 

14)2 

• 

X 

A 

X..  „ 

24)2 

21 

22 

24)2 

_ oi 

“ 

— “ 

the  time  derivative  of  Equation  (123)  is 


^14)2  " %2  ^24)2  ’^4'2  ^242 


(119) 

(120) 

(121) 

(122) 

(123) 

(124) 

(125) 

(126) 

(127) 

(128) 


and  after  inserting  elements  of  Equation  (127)  into  Equation  (128)  one 
obtains 


(129) 


which  relates  T(t)  to  T.^Ct)  and  T,„(t). 

(f)l  <P^ 

By  Equations  (46),  (58),  and  (59)  it  is  seen  that  T(t)  has  factors  in 
j space  and  in  n-space. 


T(t)  = X,  X/  = G-J  = G,,  G-1 
nxj  jxn 


(136) 


1 

J 


"^(^2  " ^11  %2  ■ '^^2  ^22  ~ "^(^2  ^21  "^(^2  ^12 

Contrast  Equation  (129)  with  Equation  (114).  By  Equation  (123) 

^1((.2^^^  ^2(p2^^^  ~ 

By  Equation  (119) 


%2<'> 

II 

t’l2">  ’'2t2‘“' 

’<2*2<"> 

^22^^^  ^2^2^°^ 

The  pseudo-inverse  for  full-rank  factor  is 

^2(J.2^^^  " ^2(()2^°^  ^22^^^ 
and  used  in  Equation  (130) 

►l2 


%2^^^  ■ *^12^^^  ^2^2^°^  ^2(\i2^°^  'f'22^^^ 


or 


%2^^^  = ^>12^^^  ‘f’oi(t) 


22 


(130) 


(131) 


(132) 


(133) 


Solving  Equation  (115)  and  Equation  (133)  for  ((i^^  and  (()^^  respectively 


12 


h2<t)  = T^j<t)  tjjd) 

Using  Equation  (134)  in  Equation  (48) 

T(t)  = CT^i(t)  ^j^/t)  + (t)„  T(o)][(()^^  + T^,(t)  ((),,  T(o)] 


-1 


ai' 


11  <1,2' 


r22 


(134) 


(135) 


ifi 


■ ;( 


One  can  also  obtain  lower-upper  triangular  factors,  Lancos  factors,  and 
eigenvalue  factors  as 
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T = I.L 


(137) 


= L D L = UA.V'- 
u u £ 

= eae" 


etc;  the  Lancos  factors  are  orthogonal  factors  also  known  as  the  singular 
value  decomposition.  The  derivatives  of  the  above  factors  yield  interesting 
relations,  some  of  which  are  obtained  in  later  sections. 

By  Equations  (115)  and  (133)  it  is  seen  that  T and  T „ are  only 
dependent  on  the  submatrices  of  , hence  one  would  suspect  that  the  matrix 
Riccati's  could  be  derived  via  partitioning  of  4*  only.  By  Equation  (65) 


i = (i^,  ^3)  = A[(t)^,  ^2:1 

and 


= A <1)^ 


♦11 

It 

^11 

> 

(-■ 

1 

♦21 

^21 

CM 

CM 

< 

CM 

-e- 

— - 

= ’’♦1  ♦!!“> 

and  at  time  zero 

hence 


(138) 

(139) 

(140) 

(141) 

(142) 

(143) 


By  taking  the  derivation  of  Equation  (141)  one  obtains  the  Riccati's 
relation  of  Equation  (115) 


Dynamics  of  Singleton  Pseudo- Inverse s . The  dynamics  of  the  singleton 
pseudo-inverses  matrices  (or  duals)  are  obtained  by  Equation  (44) 


and  for  full  rank  T 


T"^  Xj  = 


or 


„-l 


a 

hh 


Multiply  Equation  (146)  by  X^ 

^ _l  * * 

Xi  T = X^  X^  X^ 

* _i  * 

Xi  T = X^ 
Multiply  Equation  (144)  by  P 


11 


"2  "ll 


T X^  P^^  = T X^ 


since 


""l^l  = "l 


By  Equations  (144)  and  (149) 


"2  ^1 


or 


4 = ^1  "2 

Equation  (148)  becomes 

= ’’ll  X2(X2!<2''" 

Using  Equation  (152)  in  Equation  (153) 

* * 

X^  T = X^ 

or  transposing 

*t  -t  *t 
X2  = T ^ ^1 

By  the  identity  relation 
1 


TT 

one  obtains 


= 1 


(145) 

(146) 

(147) 

(148) 

(149) 

(150) 

(151) 

(152) 

(153) 

(154) 

(155) 

(156) 
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0 


(157) 


■ -1  '-1 
T T + T T 


or 


*-l  -1  * -1 

»ji  « —X  “p  'p 


(158) 


By  Equation  (33) 


or 


* 1 T 

T'  = -T”-^[-TA^^  + A22T  - TA^^T  + 


- ’■■'''22  - ’■■"Si’’"'  * *12 


or  transposing 

= T~^A 


11 


"22"'  "^2 


T A^^T' 


(159) 


(160) 


From  the  general  form  of  the  Riccati  Equation  of  Equation  (33)  and 
the  dynamics  of  Equation  (3)  and  the  connection  of  Equation  (14)  we  have 
the  associated  dynamics  for  the  Riccati  Equation  of  Equation  (160)  and 
the  connection  matrix  of  Equation  (155) 


X 

^2 


"11 

^12 


21 

'22 


*t 


(161) 


The  singleton  pseudo- inverses  are  given  by  Equation  (160),  Section 
( ) as 


*t 

"1 

*t 

''2 

^-1 

"11 


.-1 

"22 


(162) 


or 


*t  - 1 

X = dia(G.t)X 
s 11 

where  the  diagonal  block  matrix  is 

I-l 


(163) 


dia(GTt)  = 

11 


"11 


^-1 

"22 


(164) 


Taking  the  derivative  of  (163) 
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r T 


and 


= d dia(GTbx  + diaCcT^X 
s ^ 

X ^ = Id  dia(G.^)  dia  G,  + dia(G.})A  dia  GK 

s j^-  XI  1 XI  J, 


*t 


(165) 


(166) 


or  in  open  form 


^1 


° ^22 


"ll  ° 


,-l 

"11 


,-l 

"22 


"'ll  "^12 


^21  ^22 


"ll  ° 


also  combining  matrices 


=u'=ii*=u''iAi 


'^22*21 ‘^11 


- — i 

; *t 

X, 

1 

^ *t 

-2 

‘=u''l2S2 


'^22'^22'*'^22*22^22 


22 


*t 


*t 


*t 


*t 


(167) 


(168) 


Dynamics  of  X Complete  Pseudo-Inverse  For  Non-Fuxl  Rank  Case.  The 
dynamics  of  the  Generalized  inverse  for  the  non-full  rank  case  of  Equation 
(20),  that  is 


where 


X = 
2nxj 


F^(t)  = 


~ — 

r*  ^ 

X, 

I 

1 

= 

Xo 

T 

2 

» .m 

Xf  = Xj^(t) 


(169) 


I 

T(t) 


has  rank  n.  The  pseudo-inverse  of  full-rank  factors  of  Equation  (169)  is 


« S- 

"l  ^ 


The  pseudo- inverse  is  given  by 

’t 


* t -1  t 

Ft  = (Ft  Ft) 


(170) 


(171) 


where 
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(172) 


F.  'F.  = (I.  T-)/I  ) = I + T‘  T 
Consider  the  derivative  of  Equation  (169) 


X = F^X^  + F^X^  = AX 


By  Equation  (3) 


Ft  F^  t A F^  X^ 


Multiply  Equation  (174)  on  left  by  X^ 


Ft  = -FtlA^t,  A^^)Ft  * A Ft 


which  is  a rectangular  matrix  Riccati  differential  equation. 


Partition  the  matrix  A as 

X 


A = 


1 

nx2n 


2 

nx2n 


and  Equation  (175)  becomes 


^ = “VXt  ^ 


One  can  take  the  derivative  of  Equation  (170) 


* »*•  * .*«  A • A 

Xi"  = X^-Ft"  " Xl'q" 


and  attempt  to  obtain  F^  from 


^ ^ = I 


or 


F^F^  tF^F^  = 0 


(173) 


(174) 


(175) 


(176) 


(177) 


(178) 


etc;  however  the  effort  becomes  difficult  and  will  not  be  pursued  further 
here. 


Adjoint  Dynamical  Systems.  Consider  Equation  (23) 


(179) 


for  the  special  case  of 
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(a  constant) 


(180) 


^2^  = SlO 


then 


and 


By  Equation  (162) 


X,(t)  = (t) 

The  time  derivative  of  Equation  (183)  is 


(181) 


(182) 


(183) 


Xj(t)  = 

By  Equation  (183)  one  has 

XJ  1 X^(t) 

and  by  Equation  (184) 


- ^ 

• 

• 

^1 

^2 

Sio^i"" 

By  Equation  (183) 

and  the  derivative  is 


(184) 


(185) 


(186) 


(187) 


By  Equation  (187)  the  package  of  singleton-pseudos  is 


^ 

r*  — 

X 

^1 

II 

X 

^1 

J. 

'*  L 

"2 

Gj-Jx^(t) 

••  — 

and  at  the  rate  level  by  Equation  (188) 


(188) 


(189) 
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Usinp  Eouations  (185)  and  (186)  in  Equation  (3) 


and  by  Equation  (161) 


Using  Equations  (189)  and  (190)  in  Equation  (192) 


By  Equation  (191) 


(190) 


(191) 


(192) 


(193) 


"l  - "^11^1  " ^2^10^1 
By  Equation  (193) 

*^210  ^1  " '^12  ^1*  ■ ^22  ®210  ^1 
or  multiplying  Equation  (195)  by  G^io  left 

= <=210  x/"  - =2^0  *22  °2W  Xi(t) 

Equating  coefficients  of  Equation  (196)  and  Equation  (194)  one 
obtains 

^12  ~ *^210  ^12  *^210 


and 


t - e -t  i P f 
22  ■ ^210  11  ^210 


or  transposing 


“^22  " *^210  '^ll  °210 


(194) 


(195) 


(196) 


(197) 


(198) 


(199) 


*•  f ^ • 
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also  by  Equation  (191) 
*t 


SlO  "l 


A^i  + A22 


*t 


and  by  Equation  (193) 


• *t  t *t  t -t 

"1  = -'"n  ^1  ^ '^21  Sio  "1 


Multiply  Equation  (200)  on  left  by  G 


-1 


210 


• *t  -1  -1 

X,  = G^,;:  A^,  X,  + G„,:  A„„  G, 


*t 


‘1  210  21  1 210  22  210  1 
Equating  coefficients  of  Equations  (201)  and  (202) 


A^=r“^A 
21  210  21  ^210 


and 


"^11  ~ SlO  ^22  SlO 


Using  Equations  (197),  (199),  and  (203)  in  Equation  (193) 


*t 


“t 


-A 


11 


^210  ^21  SlO 


^210  ^12  *^210 


-Sio  ‘il  Sw 


*t 


SIJ  "1 


The  two  systems  are  now  Equations  (191)  and  (205);  there  is  only  one 
independent  variable  X (t),  hence  the  coupled  system  can  be  written  by 
use  of  Equation  (198)  for  in  Equation  (191) 


““  — 

— 

— 

— 

^1 

^2 

^1 

_=210 

^21 

-Sio 

®210 

®210 

If 


®210  ■ ^2  ^1  “ ^ 


then 


T(t)  = (Xj^Xj^^)"^  = G"^ 


By  Equation  (197) 


^12  ' ^21 
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is  symmetric  and  by  Equation  (203) 


^21  ^21 


(210) 


is  symmetric,  hence  by  Equation  (206) 


where 


— — 

"1 

^1 

"12 

"1 

• sVt 

"1 

''21 

*t 

^1 

— 

(211) 


*t  -1 

Xi  = 


(212) 


Clearly  now  by  the  Riccati  equation  Equation  (33)  one  obtains 


" = -T\l  - ^1  T - "'^12  " ^ ^^21 


(213) 


Likewise  any  Riccati  equation  of  the  form  of  Equation  (213)  implies  a 
coupled  dynamical  system  like  Equation  (211). 


there  are  a number  of  other  interesting  constraints  one  could  pursue  at 
this  point,  some  of  which  are  presented  below: 


What  is  the  nature  of  things  geometrically,  etc.,  when 


X2  Xi  = 0 


(214) 


What  if  the  metric  equals  the  G^^^  metric,  then 


or 


X2  x^  = T X^  X^ 


or 


G22  = T G^^  T = G^^ 


(215) 


a congruent  auto-morph  constraint. 


Clearly  for  the  case  of  Equation  (212) 
,-l 


S2  - "11 


(216) 


What  if 


M , till  uj 


a constant  like  a rigid  base  or  in  particular 
°110  ” ^ 

An  ON  base  (time  varying).  For  case  of  Equation  (211) 

rr 


A = 


^11 

^2 

^21 

"l2 

♦ ‘ 

= A 


H 


21  21 

A = A,  is  said  to  be  Hamiltonian  via  Bucy  pp  (67) 
n 

Ah  A 

e K 


(218) 


is  sympletic. 

Rectangular  Matrix  Riccati.  Consider  the  partitioning  of  Equation  (3) 
for  the  two  cases:  TiJp^^fp^and  (2)  p^^  P2. 


Case  (1):  for  pi < P2  Equation  (14)  becomes 


P2xj  P2’'Pl  p^xj 

and  for  j=l  a single  vector.  Equation  (20)  becomes 


(219) 


1 X 
X 


(t)X 


I I x(t^^  = F^(t) 

(Pl>fP2>Pi 


Pl^Pl 


T(t) 

P2"Pl 


Equation  (41)  becomes  for  the  sequence 


X^(t)" 

r 

I 

X^(t)  = 

X2(t) 

t 

T(t) 

11 

'*’12 

21 

<('22 

I 

T(o) 


X^(o) 


(220) 


(221) 
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I f rturt  Ilf  iTaaAiiaiaifai 


One  can  interpret  Equation  (220)  as  all  solutions  to 


X = AX 


such  that 


x(t^^  = F^(t)  x(t^^ 


or  also 


x(t^2  = T(t)  x(t)]>^ 


(222) 


(223) 


(224) 


which  states  that  the  second  p2-tuple  of  coordinates  are  a function  of  the 
first  pi-tuple  of  coordinates. 

By  Equation  (220)  we  are  talking  about  the  solutions  to  Equation  (222) 
constrained  to  lie  in  the  time-varying  subspace  of  dimension  pi  or  less  (or 
spanned  by  the  pi  column  vectors  of  F'i'(t).  The  dynamics  of  F(t)  is 


F^(t)  = 


r-“  1 

LT(t)J 


(225) 


or  by  Equation  (175) 

^ - ''t  ''t  " "'t 

For  the  case  F^(t)  is  full  rank,  we  have  a time-varying  base. 
By  Equation  (28) 

Xi(t)  = r ^ 1 

I T(t)  ! 


(226) 


(227) 


X^  = V^(t)  X^(t) 

Pl-Pl 


(228) 


I 


Note  the  initial  conditions 


Xj^(O) 

x„(o) 

— 

“l 

^0 

2 

Xj^(O) 


(236 


are  fully  determined  by  the  rank  of  X^CO)  since  Fq  always  full  rank. 


Suppose  j=pi  and 


I 

r 

and 


X^(0)  = 

PlXPl 


I 

P^xp 


Equation  (238)  becomes 


(23?: 


Xi(t)  = V^(t)  X^(t)  (238 

PlXPl 


subject  to  Equation  (237)  or  a fundamental  solution  in  p^  space.  One  can 
make  further  studies  on  the  rank  of  T(t)  and  the  dynamics  of  T*(t)  as  well 
as  the  dynamics  of  the  two  projectors 


^(TT*)  = ? 
dt 


and 


^(T*T)  = ? 
dt 

Case  (2):  if  one  wants  to  partition  Equation  (3)  such  that  pi<P2 
reverse  the  roles  by  obtaining 

X^  = L X2  (239 

p xj  Pi''P2  p xj 


(241) 


and  obtain  corresponding  results  except  one  obtains 


^1^2 


L 

P1XP2 


and  selects  initial  conditions  or  X2(t)  such  that  X2(t)  is  always  full  rank. 
Clearly  L will  be  different  from  T.  Another  interesting  observation  is 
when 


and  by  Equation  (227) 

X^  = T(t)  X^(t) 

That  is  the  velocity  matrix  is  T(t). 


(242) 


(243) 


28.  HOMOGENEOUS  SYSTEM  DIAGONALIZATION  OR  DECOUPLING  OF 
TRIANGULAR  SYSTEMS 


There  are  a number  of  methods  for  block  Diagonalizing  a matrix,  a few 
of  which  will  be  derived  in  this  section.  The  papers  of  Friedland  are  per- 
haps best  known  on  bias  estimation  to  the  practitioner  of  the  estimation  game 

The  first  case  to  be  considered  will  be  a block  triangular  system,  with 
the  generalization  case  deferred  to  a later  part  of  the  section. 

Consider  the  system  of  Eq  (3)  sec  (2)  with 


as  before  for  sequences  of  trajectores,  these  become  matrix  equations  or 


If  one  wants  to  solve  the  particular  uncoupled  system 


and  generate  the  solution  as  a linear  combination  of  the  uncoupled 
solutions. 


"l  = ^1  " "l2  J2 
= (I,  f_)|y,~| 


and 


X2  = ^2  = (0,  I) 


Packaging  Eq  (6)  and  Eq  (7)  and  calling  this  case  i 


- 

— — 

’•  ■*“ 

X 

I F,„ 

Y, 

1 

12 

1 

^2 

0 I 

^2 

_ _ 

or 


where 


X = F Y 

2nxj  2nx2n  2nxj 


clearly 


12 


,-l 


12 


Taking  the  derivative  of  Eq  (5) 


(7) 


(8) 


(9) 


(10) 


(11) 


. J 


or 


Xi  = Yi  + ^2  ^12  '^2 

Ey  Eq  ( 3 ) and  Eq  ( 4 ) 

(Aii,Ai2) 


(A^1,A^2^ 


^1 

- "11  ^ 

1 

+ (F 

^2 

- 

— 

— - 

I 

"12 

0 

I 

^2 

J 

^^11’ ^12  '*■  ^12  ^22^ 


(12) 


(13) 


(14) 
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F 


then  by  Eq  (8)  for  j=l 
x( 


or 


If  we  apply  the  constraint  that  ^ constant,  that  is 

= 0 

then  Eq  (16)  becomes  the  algebraic  Riccatti  equation 
° ^ ^11^12  ■ ^12^22  ^12 


(24) 


(25) 


(26) 


(27) 


solutions  of  which  will  be  discussed  in  section's  using  Kronecker  Products 
etc. 

Observe  that  X may  be  obtained  by  solving  the  upper  Block  triangular 
system  of  Eq  (3) 

or  the  block  diagonal  system  of  Eq  (4)  which  has  solutions 


(28) 


where 

i 


A, , t 

11 

e 0 

Y^(o) 

A„-t 

n 22 

0 e 

Y2(o) 

At 


2^2 


e " = I + At  + At  + 
2! 


plus  solving  the  matrix  Riccatti  Differentenal  equations  of  Eq  (16)  and 
obtaining  X from  Eq  (9)  namely 


X = FY 


(29) 


One  can  also  solve  the  algebriac  Riccatti  Equation  of  Eq  (27)  to  obtain 
a solution. 


Case  ii. 


Consider  next  the  solutions  for  X put  together  via 


Xi  = t ^2 


"2  = °"l  " ^22  "2 


(30) 


or  matrix-wise 


The  inverse  is  easily  computed  as 


The  matrix  B of  Eq  (19)  is 
B = -F"^  F + a F 


If  one  selects  to  be  a constant 


^11  ^12  '''  ^11  ^12  ^12  ^22  “^12  ^22  ^22  ^22 
B = (40) 

° ^22  ^22  ^22 

and  the  Riccatti  equation  is 

' -1 
F =:A  F +A  F -P  F A P 

12  “ll  "l2  *'12  ^22  ^12  22  ‘^22  ^22  (41) 

observe  that  by  Eq  (40)  if  one  selects  the  matrix  to  be  the  matrix  which 
diagonalizes  A 


®22  ” ^22  ^22  ^22  ®22  ^2 
then  the  second  block  of  is  uncoupled,  also 


e V Y,(o] 


Case  (iii)  consider  the  case  of 


Xi(t)  - Y^  = Y^ 


X2(t)  = Y^ 


hi  ' 


The  inverse 


I F~^  -F~^ 

11  11 


Equation  (33)  for  this  case  becomes 

1 hi^~  hi  ^11  hi  “ ^21  hi^  hi^hi  h2  ~ ^21  " ^221 


^21  hi 


hi  ^ h2 


T 


and 


^21  = ° 


(48) 


Using  (48)  in  the  element  of  Eq  (47) 
^11  ^ ^12  “ ^22  ^ ° 


or 


and  the 


^12  ^22  ^11 


®22  ^22 


(49) 


and  the  one-one  element 

.-1 


®11  = "ll  ^-"ll  " ^1  ^1^ 


^11  ^11  ^11  ■ ^11  ®11 


(50) 


Clearly  by  Eq  (49)  the  transformation  of  Eq  (45)  will  work  only  for 
the  special  case  of 


A = 


11 


^22  ^11 


22 


(51) 


r I 


Return  now  to  case  one . By  way  of  summary:  by  Eq  (3)  and  Eq  (33) 
sec  (2) 


X = 


^11  ^12 


22 


X2  = TX^ 

T = A22  T - T All  - T ''12  T 


(52) 

(53) 

(54) 


By  Eq  (9)  and  Eq  (10) 
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and  by  Eq  (4) 


Y = 


All  ° 


22 


and  by  Eq  (16) 

F 

The  Y connection  matrix  is 


12  ■ ^11  ^12  ■ ^12  ^22  ^12 


Y2  = QYi 


(a 


with 


« = *22  <5  - 0*11 


(b 


The  matrix  Riccatti  relation  of  Eq  (57)  implies  a dynamical  system 

(b 


"2  = hi  ^ 


where 


■'ll 


'12 


'22 


= CZ 


(e 


Taking  the  derivative  of  Eq  (60)  and  using  Eq  (61)  one  finds  that 


By  inspection  we  see  that 


C = L A L 
c c 

where  the  linear  convolution  matrix  is 


Thus  if  one  defines  the  transformation 


Z = L X 
c 

then 

Z = L A L X 
c c 

also  by  Eq  (55) 


or 


Z = L F Y 
c 


where 


t 

zy 


L F 
c 


By  Eq  (53) 


X = 


and  by  Eq  (60) 


Z = 


(64) 

(65) 

(66) 

(67) 

(68) 

(69) 

(70) 

(71) 

(72) 

(73) 
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Using  Eq  (72)  and  Eq  (73) 

[:] 

^1  = 

0 I 

1 0 

I 

T 

_ 

M •• 

By  Eq  (67) 

= "2 

Eq  (75)  in  Eq  (74)  yields 


— - 

r— 

— 

— — 

I 

0 

I 

I 

^12 

I 

0 

T 

Multiply  Eq  (76)  on  right  by 


12 


f = 


or 


or 


F T 
12 


F T = I 
12 


0 I 

1 0 


which  implies 

"12 


I 


(74) 


(75) 


(76) 


(77) 


(78) 


(79) 


(80) 


hence  we  see  that  the  transformation  that  diagonalizes  Eq  (52)  is  given  by 
Eq  (55)  and  Eq  (80)  as 


(81) 


I T"^ 

0 I 

a rather  interesting  result. 

The  transformation  between  the  Z and  Y systems  are  now  readily  obtained 
from  Eq  (67)  and  Eq  (55)  as 


Z = L FY 
c 


(82) 
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Solution  to  the  Upper  Triangular  Dynamical  system  via  The  Diagonal  System 
and  a matrix  Riccatti  Dynamical  System. 


This  section  presents  4 alternatives  to  solving  the  upper  triangular 
linear  dynamical  system  given  by  Eq  (3)  as 

Pii  ki  Ri 


subject  to  the  initial  conditions, 

Xl<°>  = *10 


= Xjij  (87) 

Method  one  consists  of  solving  Equation  (86)  and  (87)  by  any  standard 
numerical  methods  or  continuious  analog  methods. 

Method  two.  Since  the  second  matrix  of  Eq  (86)  is  uncoupled  from  the 
first  matrix,  one  has 

^2  ^ ^22  ^2  (88) 

with  solution 

Xj  = X2<°>  = '•a,,  *20  (89) 

Equation  (89)  can  now  be  used  in  the  first  equation  of  Eq  (86)  as 
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nimiiiiir--igrimrl 


(90) 


^1  ^11  ^12  ^2 


• ^22^ 

"2  = ^ ^12  ^ "20 


(91) 


which  is  a non-homogeneous  linear  differential  equation,  the  solution  of 
which  is  deferred  until  a later  section. 


Method  three . This  method  used  the  solution  of  the  uncoupled  matrix 
of  Eq  (89) with  equation  (14)  sec  (2) 


X 


1 


(92) 


or 


X 


1 


A 

e 


22 


t 


X 


20 


and  by  Eq  (85) 


+ A 


12 


(93) 


(94) 


with  initial  condition 

T‘^(o)  = X^(o)  X2*(o)  (95) 

or 

T~^(o)  = Xj^(o)  X2(o)  (X2(o)  X2^(o))"^  (96) 

This  scheme  of  things  requires  the  solution  of  non-homogeneous  "bi-linear" 
(missing  the  guadratic  term)  Riccatti  Equation,  Clearly  Eq  (94)  is  more 
difficult  to  solve  than  Eq  (91)  a method  for  solving  Eq  (94)  is  given  in  a 
later  section. 


Method  Four.  This  method  is  a variation  of  the  last  method  and  is  the 
basis  to  the  bias  estimation  scheme  of  Friedland  in  reference  (6).  Levin 
also  in  reference  (51)  relates  two  solutions  of  the  same  Riccatti  differential 
equation  to  a new  system,  the  details  of  which  are  disscussed  in  more  detail 
in  section  (33) . 


The  methods  obtains  the  transformation  Eq  (55) 


which  diagonalizes  Eq  (86)  in  the  special  way  of  Eq  (56) 


(97) 


All  ° 


"22  ^2 


The  solutions  to  the  uncoupled  systems  are 
A t 

Yl  = e Yl(o) 

^22^ 

^2  = ® ^2^°^ 

where  the  initial  conditions  are  by  Eq  (97) 


Y^Co)  = X^Co) 


Xi(o)  - Yi(o)  = Fi2(o)  X^(o) 


Fi2(o)  = Xi(o)  X^(o)  - Y^(o)  X^(o) 


By  Eq  (95) 

Fi2(o)  = t“^(o)  - Yi(o)  X*(o) 

By  Eq  (104)  we  see  that  there  are  two  free  choices  of  i.c.  matrices 
Fi2(o)  and  Yi(o).  For  example  if  one  selects, 

Yi(o)  = 0 


Fi2(o)  = t"^(o) 

and  the  case  becomes  the  same  as  method  three  for  then 

However,  for  all  cases  in  which 
Yi(o)  ji  0 


Fia't)  ^ T-4t) 

As  an  example  suppose 


(100) 


(101) 


(102) 


(103) 


(104) 


namely 


(105) 


(106) 


(107) 


(108) 


(109) 
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Yj^(o)  = Xj^(o) 

then  by  Eq  (103) 


(no) 


F,j(o)  = 0 


(111) 


Two  solutions  to  the  same  matrix  Riccatti  differential  equation,  as 
shown  in  section  03) , are  related  as 


Fi2(t)  = T"^(t)  + N(t) 


(112) 


where  by  Eq  (16) 


^12  ■ ^11  ^12  ■ ^12  ^22  ^12 
and  by  Eq  (97) 

= *11  *22  *12 
Tasking  the  derivative  of  Eq  (112) 

• •_! 

Fi2-T  =N 


(113) 


(114) 


(115) 


Using  Eq  (113)  and  Eq  (114)  in  Eq  (115) 


N = - NA22  (116) 

which  is  a homogeneous  "bi-linear'*  matrix  differential  equation.  Since  the 
i^strix  Riccatti  Differential  Equation  of  Eq  (116)  implies  a dynamic  system 
connection  as 


W2  = N 


then  by  Eq  (3),  Eq  (14)  and  Eq  (33)  of  sec  (2)  one  has 
A22  0 

0 A 


11 


or  for  full  rank  W^(t) 


N(t)  = W2(t)  W^(t) 


(117) 


(118) 


(119) 


or  the  matrix  factors  of  N have  uncoupled  dynamics,  Eq  (115)  and  Eq  (118) 
can  be  compared  with  Eq  (59)  and  Eq  (56).  Clearly  since  matrix  factors 
are  not  in  general  unique.  Let  N(t)  have  matrix  factor  defined  as 
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as  1^.1  mi  mpw 


or 


Uj^(t)  = N(t)  U^Ct) 


= N(t) 


Take  the  derivative  of  Eq  (120) 


= N U2  + N U2 


where 


— 

'^1 

°11 

°12 

^1 

"2 

°21 

°22 

"2 

or 


or 


<‘’11"  ♦ ’ “"2  ♦ * ‘>22^“2 


N = - N(Dj^N)  - NDjj  + 


and  by  Eq  (116) 


A^lN  - NA22  = '^ll'^  - N°22  - ^°21  " °12 


which  implies 


D = 


or 


All  ° 


22 


"11  ° 


22 


u. 

1 

. 

(120) 


(121) 


(122) 


(123) 


(124) 


(125) 


(126) 


(127) 


(128) 


Thus  we  so  that  by  taking  solution  differences  and  selecting  factors  of 


r 


UjCt)  = e U^(.o)  = U2(o) 


and  the  psuedo-univses  is 

* ft  ~^22^  X -1 

Uj  (t)  = Uj  (o)  e = Uj  (o)  (jl^ 

hence  by  Eq  (132);  (130)  and  Eq  (121) 

A t -A^„t 

N(t)  = e Uj^(o)U2  (o)  e 

N(t)  = (j).  (t)  N(o)  ({'.■^(t) 

^11  ^22 

and  the  initial  conditions  are  by  Eq  (132) 

N(o)  = U^(o)  U2‘‘(o) 
and  by  Eq  (112) 

N(o)  = - T‘^(o) 

By  Eq  (103)  and  Eq  (95) 

N(o)  = X^(o)X2(o)  - Y^(o)X2(o)  - X^  X* 


N(o)  = -Yj^(o)X2(o) 

By  Eq  (134) 

N(o)  = U^(o)  02(0) 

thus  if  the  diagonal  U system  is  initialized  with 
U^(o)  Y^(o)  Xj^(o) 

□2(0)  -Y2(o)  -X2(o) 

then  the  U system  is  time-wise  related  to  the  Y system  as 


u^(t)  K(t) 


(130) 


(131) 


(132) 

(133) 


(134) 


(135) 


(136) 


(137) 


(138) 


(139) 


(140) 


Using  Eq  (139)  in  Eq  (136) 

^11^  * ~^22^ 
N(t)  = -G  ^ X^(o)X2(o)  e 

and  using  Eq  (141)  in  Eq  (112) 


F (t)  = T"^(t)  - ({l,  (t)  X^(o)  X*(o)  ^(t) 

1^  All  ^ ^ ^22 


By  Eq  (97) 

X_^(t)  = Y^(t)  + ^2^^^ 

using  Eq  (142)  in  Eq  (143) 

X (t)  = -T‘^(t)  (j).  (t)  X-(o) 

^22  ^ 

which  is  the  same  as  Eq  (93)  under  method  three. 


(141) 


(142) 


(143) 


(144) 


The  Diagonalizing  Transformation  via  Kronecker  Matrix  Product  Solutions, 
By  Equation  (19)  the  dynamic  system  matrices  are  related  as 


B=-FF+F^AF 

or 

F = AF  - FB 
By  Eq  (33)  sec  (g) 

vec  (AF)  = (ISA))  vec  F 
and  by  Eq  (49)  appendix  g 

vec  FB  = (B^fll)  vec  F 


or 


vec  F = ^ISA)  - (B''^ai)  ] vec  F 


(145) 


(146) 


(147) 


(148) 


(149) 


which  is  a linear  matrix  differential  equation  in  the  tensor  product  space 

of  dimension  4n^,  that  is  the  2nx2n  matrix  F becomes  a column  vector  of 

. 2 

size  4n  X 1.  One  may  now  ask  whether  a transformation  which  is  a constant 
matrix  can  be  found  or 


F = 0 


(150) 
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Section  29  TRIANG13LARIZATI0N  AND  DIAGONALIZATION  OR  DE-COUPLING  OF  ARBITRARY 
NON-HOMOGENEOUS  DYNAMICAL  SYSTEM.  The  fully  coupled  non-homo- 
geneous  form  is  . 


x(2a^  =1  , * ■ - ■ " 

’''2/  \"21  "22 
A full-rank  base  change  of  the  form 

= Fy 

yields  y dynamics  by  taking  the  derivative  of  Equation  (2) 
A^  ^ ^ 


or 


or 


y^  = (-F"^F  + F"W)^  + 


y>  = By>  + f> 


where 


B = -F"^F  + f'^AF 


(1) 


(2) 


(3) 


(4) 


(5) 


(6) 


,-l 


In  the  previous  section  the  form  of  F was  taken  so  that  F could  be  easily 
obtained  from  triangular  f^rms  for  F.  Also  it  was  shown  that  via  the  matrix 
Kronecker  products  that  F term  was  not  needed.  The  Kronecker  product 
solutions  via  Equation  (149)  of  Section  (27)  can  be  difficult.  This  section 
shows  some  easily  invertible  matrix  forms  used  by  triangular  matrix  products. 

This  section  obtains  the  composite  (product)  maps  by  going  from  the  full  system 
of  Equation  (1)  to  an  upper  traingular  system,  that  is 


X ^n>  = F^yy  (2n^ 

where  the  B of  Equation  (6)  is  upper  triangular 


B 


B = 


11 


B 


12 


B 


22, 


followed  by  the  transformation  to  a new  system 

i>  = 

with  dynamic  matrix  C which  is  diagonal  that  is 


= 


'11 


(7) 


(8) 


(9) 


(10) 


22. 


••  • •• 


677 


By  Equation  (9)  in  Equation  (7) 


X tin: 


= F F = F z> 

xy  yz  ^ xz  ^ 


xz 


(11 


or 


F = F F 
xz  xy  yz 


(12 


and 


F”^  = F~^F~^ 
xz 


yz  xy 
By  Equation  (6) 


(13 


C = -F'^F  + F‘^AF 

xz  xz  xz  xz 


(14: 


and  the  inverse  matrix  F is  easily  obtained  via  Equation  (13)  as  the  product 


XZ 

of  simple  triangular  matrix  inverses, 

Returning  now  to  the  first  stage  via  Equation  (6)  with  proper  subscripts 


B 


B, 


11  12 
0 B 


1 • -1 

F + F AF 
xy  xy  xy  xy 


(is: 


22 


with 


F 

xy 


I 

.L 


0 

I 


(16: 


and 


f"^  = 

xy 


I 

■L 


0 

I 


(17] 


The  first  matrix  product  term  on  the  right  of  Equation  (15)  is 


F'^F  = 
xy  xy 


I 0 
•L  I, 


0 0 
L 0 


0 O' 
L 0> 


(18] 


The  similarity  transformation  term  is 


f"^af 

xy  xy 


A11+A12L 


12 


■L(Ajj+A^2^)+A2j+A22l‘ 


(19] 


■^12^^22. 


Using  Equation  (18),  Equation  (19)  in  Equation  (15) 


'11 


B 


12 


AJJ+A12L 


•12 


(20: 


22 


■L-L(AjiMj2^)^A21^'^22^ 


-LA12+A22 


we  find 


(21) 


(22) 


Note  that  L is  same  as  Equation  (33)  Section  (26)  that  is  the  Ricatti  equation 
for  T, 


By  the  previous  section  the  upper  triangular  system  was  transformed  to  a block 
diagonal  system  via  the  transformation 

where 


F = 
yz 


(24) 


The  dynamic  matrices  for  this  case  become 

C = + f'^bf 

yz  yz  yz  yz 

By  analogy  with  Equation  (20)  Section  (3) 


-M+B^^M-MB22'^Bj2 


22 


(25) 

(26) 
(27) 


The  B^j  are  given  in  terms  of  the  via  Equation  (21). 

The  composite  transformation  is  by  Equation  (24)  and  Equation  (16)  in  Equation 

(12) 


F = F F 
xz  xy  yz 


(28) 


or 


(29) 


(30) 


679 


with  inverse  given  by  Equation  (13) 


where  by  Equation  (20)  in  Equation  (26) 


° )- 

^ll'^^12^ 

0 

\ 0 

<=22^ 

0 

^22"^12- 

and  by  Equation  (27)  and  Equation  (20) 


(31) 

(32) 


(33) 


In  case  one  wants  constant  transformation 
L = M = 0 

and  Equation  (22)  and  Equation  (34)  become  algegraic  matrix  Ricatti  equations  or 


0 = 

(35) 

0 = (AjjH-Aj2L)H-H(A22-LAj2)Mj2 

(36) 
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Section  30  TRANSFORMATION  OF  HOMOGENEOUS  DYNAMICAL  SYSTEM  TO  BLOCK  COMPANION 
MATRIX  FORM.  This  section  considers  the  constant  transformation  to  map  the  the 
dynamical 


to  a system  in  block  companion  matrix  form 


where 

X = FY  (3) 

and 

F'^AF  = B (4) 

If  one  selects  the  constant  matrix  F of  the  form 


F =1 


vO 


12 


IT 


it  is  easily  established  that 


I 

^0 


■^12^22 
^22 


and 


or 


F"^AF  = B 


'A  -F  F~^A 
11  12  22^21 

f"^a 

22^21 


^^ir^l2^22^2p^l2  ^ ^^12"^12^22^22^^22 
^22^^21^12  ^ ^22^22^ 


(®21  h) 


Equating  the  (1,1)  elements  of  Equation  (8) 

^ir^l2^22^21  ■ ® 


or 


F f”^  - A a”^ 

'12*^22  ''ll''21 


(5) 


(6) 


(7) 


(8) 


(9) 

(10) 


and  by  Equation  (17) 


^21  ~ ^12^21  ■ ^11®22 
and  by  Equation  (20) 

^22  ~ *‘11  * ^22 
The  matrix  relations  are 


f'^af  = 


^12^21  " ^11^22 


^11  ^ ^22 


When  A is  a Hamiltonian  matrix  Section  p7  ),  or 


Then  by  Equation  (25) 


b22  = a - a = 0 


c)  i)  (y 

(O-G) 


We  see  that 

**  2 

y = (bc-a  )y  (31 

or  the  system  has  no  damping  or  velocity  dissipative  term,  that  is  the  system 
of  Equation  (31)  is  an  oscillator  when  bc-a  has  the  correct  sign  (negative 
feedback) . 

At  the  matrix  level  the  condition  is  by  Equation  (20) 

®22  ~ ® “ ^21^11^21  ^ ^22 
and  for  the  matrix  Hamiltonian  case 

^22  ~ ^11 


A A a”^  = A 

21  ir21  11 


Section  31  NON-HOMOGENEOUS  DYNAMICAL  SYSTEM  X = A X + F(t) 


This  section  considers  the  dynamical  system 

^ (1) 

2nx2n 

and  a sequence  of  j systems 

X = AX  + F (2) 

2nxj 

Two  different  methods  of  solution  will  be  presented.  The  homogeneous  system  is 
obtained  in  Section  (25)  as 

i = A<t>  (3) 


where 


5 


1 


.A 


which  is  the  "apparent  velocity"  as  observed  by  an  observer  or  the  4>  basis. 
Integrating  Eq.  (12) 


= x*^(to^  + J <f  ^(T,to)f (t^  dt 
to 

By  Eq.  (16) 

x(to)^  = Ix‘*’(to)^ 
hence  Eq.  (13)  is 

x‘**(t)^=  x(to)^+  f 4)  ^(T,to)f(T)NdT 
to 

Using  Eq.  (15)  in  Eq.  (6) 

,t  _ 

x(L^  = 4)(t  ,to)x(to)'^  + <t>(t,to)  I <t>  (T,to)f(T)^  dT 

■'  to 


Using  transition  matrix  properties 
4)'^(t,to)  = <l>(to,t) 

<|)(t,to)<l>  ^(l,to)  = 4'(t,to)t|5(to,T)  = 'l>(t,T) 
hence  Eq.  (18)  in  Eq.  (16) 

ft 

x(t)^=  <l>(t,to)x(to)^  + j (t)(t,T)f (x)^  dl 

to 

Eq.  (17)  and  (18)  are  obvious  from 

4>(t,to)  =e 

»'‘(t,io)  =e  =e 

and 

4.(t,t)  =e  e A(to-T)  , e A(t-i) 

By  Eq.  (22)  in  Eq.  (19) 

x(t^  = <t'(t,to)x(to)^  + J 6 ^^f(T)^  dt 


(13) 


(14) 


(15) 


(16) 


(17) 

(18) 


(19) 


(20) 

(21) 

(22) 


(23) 


The  convolution  integral  of  Eq.  (16)  was  very  naturally  obtained  in  the  •I'  base. 


Section  32  VARIANCES  RELATED  TO  x) . = Ax  V + u^. 

/j  Consider  the  sequence  of  j 

continuous  time  trajectories 

X = AX  + U (1) 

nxj  nxj  nxj 

where 

Ul"  j S = (2) 

mxj  ' 

Taking  the  arithmetic  mean  of  Equation  (1) 

"*>  = “t'x)  + t'u) 

Subtracting  Equation  (3)  from  Equation  (1) 


X - M > < j 1 = X = AX  + U (4) 

nxj 

where 


The  derivative  of  Equation  (12)  is 

— = <1>Z^<P  + 

XX 

Now  by  Equations  (11)  and  (12) 

and 

<1.  . 

The  derivative  of  Equation  (10)  is 
X = w'*’  + (tX*** 

Using  the  relation 
= A <t> 

and  Equation  (8)  in  Equation  (16) 

AX  + U = AM*^  + 4>x‘*’ 
or 

x‘'’(t)  = 4>‘^t,to)U(t) 

Integrating  Equation  (19) 

X*^(t)  = x'^(t,o)  + f 4)  ^(T,to)U(t)dT 
w'  to 


( 

( 

( 

( 

( 

( 

( 


where  by  Equation  (10) 

x''’(to)  = X(to)  ( 

Using  Equation  (19)  and  (20)  and  their  transposes  in  Equation  (15)  the  first 
term  on  the  right  is 


X'^(t)x‘'’'‘’  = <|)"^t,to)U(t)[X(to) 


4>“\x,to)U(t)dT]^ 


(2 


Jto 


x'f’x'^T  = (t)"^(t,to)U(t)[x’^(to) 


+ ! U^(T)4)‘^(T,to)dT 

Jto 


or 


x't'x'^’'^  = <D‘^(t,to)  U(t)X^(to)  + 


<l>  \t,to)  J 


U(t)u'^(T)4>’'^(T,to)dT 


(? 


to 
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If  the  noise  is  serially  - uncorrelated  and  zero-mean,  that  is 

E(u(t))  <u(t))  = U(t)u'^(t)  = Q(t,t)6(t-t)  (2f 

If  we  assume  further  that 

E(u(t))>  ^x(to)  1 = 0 = L)(t)X^(to)  (26 

that  is  the  initial  state-errors  are  uncorrelated  with  respect  to  the  process 


Using  Equation  (25)  and  (26)  in  Equation  (24) 


= (})‘^(t,to)  ! Q(t,T)6(T-t)<J)‘'^(T,to)dt 


x't'x't'T  = (|)''^(t,to)Q(t,t)<l>"'’^(t,to) 

Clearly  the  second  term  of  Equation  (16)  is  the  transpose  of  Equation 
= 4."\t,to)Q(t,t)(l>''^(t,to) 

Combining  Equation  (28)  and  Equation  (29)  in  Equation  (16). 

**’  = <)>  \t,to)Q(t,t)(l)”^(t,to) 

Using  Equation  (30)  and  the  relation 
i = A<1> 

in  Equation  (4) 

~~  = + Q(t,t)  + 

By  Equation  (13)  in  Equation  (32) 

— = AZ — + Z^ — A^  + Q(t) 

which  is  the  matric  Riccati  differential  equation  of  the  variance. 


(28)  or 
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By  Eq.  (33)  Section  (6) 


T = -TA^^  . A22T  - TAj^T  + A^^ 


and  by  (159)  Section  (6) 


Both  equations  are  non-honiogen,eous  and  quadratic 


Suppose  we  have  one  solution  to  Eq.  (3)  T^(t)  with  i 


T.(to)  = T. 

1 10 


and  want  a second  solution  T (t)  with  initial  condil 


Ti^j(to)  - 


obtaining 


T.^l(t)  = T.(t)  + N.(t) 


The  difference  is 


= huh 


and  the  derivative  is 


N.  = - T. 

1 1+1  1 


Using  Eq.  (3)  in  Eq.  (9) 


^i  ‘^i+1^11  ^22^i+l  ‘ ^i+1^12^i+l  ^21 


This  section  discusses  some  interesting  relations  i 
solutions  of  the  matrix  Riccatti  Differential  Equat 
(51)  drives  some  of  these.  Friedland  applies  some 
Estimation  in  reference  (32).  Lainiotis  in  referen 
methods  to  obtain  what  he  calls  partitioned  Riccatt 
taining  the  generalized  Chandrasekhar  relations.  C 


Section  33  DIFFERENCES  OF  MATRIX  RICCATTI  SOLUTION 


or 


Ni  = -(Ti^l-Ti)A„+A22(Ti,i 


Ti) 


(11) 


■^i+1^12^i+l  * ^i^l2^i 


Using  Eq.  (7)  in  Eq.  (11)  quadratic  term 


N.  = - (T.  ♦ »i)A,2(Ti  * ♦ T.A^^T. 


(12) 


= -N,A„  . A^^N.  - T.AjjT.  - T.Aj^Ni  - ".A^^T.  - ll.A,^"!  ' Via’’)  «« 


or 


N.  = -».(A„  . A.^T.)  * (A^j  - T.A,2)N.  - N.Aj^Ni 


(14) 


which  is  homogeneous-quadratic  in  N^(t).  Or 


"'i  = -^i«ll  " Vi  - ^^12^1 


(15) 


which  implies  a dynamical  system  by  Eq.  (33)  Section  (2) 


y>i 


>y>' 


B 


11 


-A 


12 


22 


y>- 


(16) 


(!>. 


where 


®11  = ^1  " "^12^1 


®22  ^22  " ^i^l2 


(17) 

(18) 


where 


y>2  = Njy'i 


(19) 


or  for  a sequence  or  package  of  trajectories. 


^2  = ^i^l 


(20) 


with  N^(t)  factors 


Ni(t)  = Y2y^ 


(21) 


The  inverse  Riccatti  relation  is 


♦-1  -1  -1 

N.  = -N.  N.N. 

1 111 


(22) 


or 


N-'  = b„n:’  - b:'b22  4 A„ 


(23) 


which  is  non-homogeneous  bi-linear. 


If  now  one  has  two  solution  of  Eq.  (23) 


"111  - "‘1'=  "i<'> 


then 


M,(t)  = rjj  - li-’ 


or 


Mi(t)  = 


M.B22 


which  is  homo-geneous  non-quadratic.  Eq.  (26)  implies  a dynamical  system 


Z> 


'll 


22' 


(:::)■  c 


22 


with  connection 


z>  = «.z>, 

^11  “ ®22  “ ^22  ■ ^i^l2 
^22  ■ ®11  ^ ^11  ■ ^i^l2 


(24) 

(25) 

(26) 

(27) 

(28) 

(29) 

(30) 


The  dynamics  of  successive  differences  and  inverses  are  applied  in  following 
the  obtain  solutions  to  Riccatti  equations  in  terms  of  a controlled  initial 
condition  solution.  Some  of  these  variations  are 

T = 

T = -TA^^ 

T = -TA^^  . A22T 
T = -TA^2T 
T = A22T  - TA^^T 
f = -TA^j  - TAj^T 
f = A22T  - TA^j  - TA^^T 

all  of  the  above  are  homogeneous  and  linear,  bi-linear,  and  or  quadratic. 

The  non-homogeneous  cases  are 

^ ^ ^22^  ^ ^21 
f = AjjT  + A2J 

f = "TAj2T  + A2J 

f = -TAjj  + A22T  - TAj2'^  ^2^ 
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Jsi, 


Section  34  SOLUTIONS  OF  MATRIX  RICCATTI  VIA  DIAGONALIZATION  TECHNIQUES 


This  section  draws  upon  the  relation's  of  section  2 for  block  diagonalizing 
a system  to  obtain  solutions  to  the  matrix  Rlccatti  differential  equation. 

Suppose  we  start  with  the  given  Rlccatti  Eq.  (3)  section  31 

T = + ^27^  ~ ^^12^  ^21  ^ 

with  i.c.  of 

T(o)  = To  (; 

Eq.  (1)  implies  a coupled  homogeneous  system  given  by  Eq.  (1)  of  section  31, 

('■)•(■“  ‘■1C‘) 


^2  = '^'^1 


or  T(t)  has  the  matrix  factors. 


T(t)  = X^Xj^* 


By  Eq.  (30)  sec  28 


L(t)  LMfl 


where  by  Eq.  (33)  of  sec  28 


All  + ^12^ 


^22  " ^12 


Z2  =•  S(t)Z^ 
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i 


f.' 

[ 

1 


and  by  Eq.  (16)  we  see  that  M(t)  is  a non-homogeneous  system  and  Eq.  (17) 

Is  simplified  and 

M(t)  = $ (t)  ‘^(T,  o)A  ♦ (T,  o)dT]r^  (21) 

C22  J O C22 

. } 

We  next  seek  the  connection  between  the  matrix  S(t)  and  the  matrix  T(t). 

By  Eq.  (4)  and  Eq.  (9)  in  Eq.  (6) 


o-  ■ c :.)  0 


(22) 


or 


= (I  + MS)Zj^ 


(23) 


Using  Eq.  (23)  in  the  second  element  of  Eq.  (22) 

TXj  = T(I  + MS)Zj  = [l  + (LM  + I)S  ] Z^  (24) 

or 


T(I  + MS)  = L(I  + MS)  + S (25) 

or 

T = L + S(I  + MS)“^  (26) 

If  we  set 

N = S(I  + MS)“^  (27) 

then 

T - L + N (28) 

which  is  of  the  form  of  Eq.  (7)  of  sec  31. 

Eq.  (23)  also  be  written  as 

T » L + (L  - T)MS  + S (29) 

or 

T - L - NMS  + S (30) 

which  implies  that 

N - -NMS  + S (31) 


[ 
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which  agrees  with  Eq.  (27). 

By  Eq.  (30) 

T = (I  - NM)  S + L 

we  see  by  Eq.  (9) 

• • • 

= SZj^  + SZj^ 

or  by  Eq.  (8)  In  Eq.  (33) 

(A22  - ^22^^  " 

or 

(A22  - LAi2^^  ■ ^ 

By  Eq.  (14)  sec  7 

N = -N(Aj^^  + Aj^2L)  + (^22  ~ ^12^”  ~ ”^12^ 
which  is  homogeneous  quadratic  in  N. 

The  initial  conditions  by  Eq.  (30) 

To  = Lo  - NoMoSo  + So 
and  by  Eq.  (19)  in  Eq.  (37) 

To  = So  . 


with  the  associated  dynamical  system 


A,,  0 WX 


with 


0 


Aj  j(t-to) 


and 


and 


* = e 


x,(t)  = <J)  X/0) 

* A ^ 

^11 


and 


A22(t-to) 


<j)  = e 

A. 


- <t>  (t)X  (0) 
*22 

The  generalized  inverse  is 

.-1 


Xj(t)  = Xj(0)<J) 


11 


By  Equation  (2)  T(t)  has  factor 
T(t)  = X2X*(t) 


or 


or 


T(t)  = <i>  (t)X2(0)Xj(0)<J)‘^  (t,to) 


22 


'll 


T(t)  = 4)  (t)T(0)<l>‘^  (t) 


22 


11 


(12) 

(13) 

(14) 

(15) 

(16) 

(17) 

(18) 

(19) 

(20) 
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Section  36  SOLUTION  OF  T - “TAj^^^  '‘’  ^22  ^21 

The  solution  of  the  hi- linear  non-homogeneous  matrix  differential  equation  of 
the  form 


I - -TAjj  + Ajj  It  A^j 


is  obtained.  Via  Eq.  (9)  of  the  previous  section 
A^2  = 0 


(1) 


(2) 


■M 


J 


hence  Eq.  (1)  becomes 


X, 


All  0 \ l\ 


' *21  *22' 


with 


and 


T(t)  = X2(t)X^*(t) 


T(o)  = X2(to)Xj^*(to) 


Obtain  T(t)  as  a solution  of  the  system 


- -^c*U  *22^0  *■  *21 


with 


T(t)  = T^(t)  + N(t) 


N(t)  = T(t)  - T (t) 
c 


N(t)  = -N(t)A^j^  + A22N(t) 


with  initial  condition 


N(to)  = T(to)  - T (to) 
c 


or 


and 


(3) 


(4) 


(5) 


(6) 


(7) 


(8) 


(9) 


(10) 
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T 


select 


T (to)  = 0 
c 


N(to)  = T(to) 

By  Eq.  (21)  of  section  8 


N(t)  = <f.  (t)  T(to)  (t) 

22  11 


The  system  T^(t)  via  Eq.  (4)  has  factors 


T^(t)  . [X2(t)Xj*(t)  ] 


and  initial  condition 


X2c(to)  = T^(to)X^^(to)  = 0 . 


Select 


Xj^^(to)  = X^(to) 


By  Eq.  (3)  the  first  coordinate  is  uncoupled  hence 
V /*.\  _ * A.-..(t*"to)  /4.„\ 


=e 


X^(to) 


The  second  matrix  coordinate  of  Eq.  (3)  is 

* 

X2c(t)  = 

or  by  Eq.  (18)  in  Eq.  (19) 

^2c^^^  * ^22^2c^^^  to)X^(to) 

which  is  a linear  non-homogeneous  equation  with  solution  given  by  Eq. 
section  34 


to)X^(to)dT 


The  initial  condition  by  Eq.  (16)  is  zero,  hence  Eq.  (21)  becomes 


By  Eq.  (18) 


X*^(t)  + X*(to)<t~^  (t,  to) 


Using  Eq.  (22)  and  Eq,  (23)  in  Eq.  (14) 


T (t)  = <!>.  (t,  to)rrj  (T,  to)A  $ (T,  to)dT  $ (t,  to) 

c A-„  TJto  A - 21  A J A 


Using  Eq.  (13)  and  Eq.  (24)  in  Eq.  (7) 


T(t)  = $ (t)T(to)4>  (t,  to) 

22  11 


+ <t  (t,  to)/^^$-J  (T.  to)A2^$  (T,  to)dT  (t,  t.) 


T(t)  = (t,  to)  rT(to)+r^  (t,  to)A„  $ (t,  to)dTl$  ^ (t,  to)  (26) 

A I JZO  A I "ll 


which  is  the  solution  of  Eq,  (1). 


Note  by  section  32  Eq.  (12) 


^A  " ^11  ^A 

All  il  A^j^ 


$ (to)  = I 
*11 


■f.  - A.,  <1>.  (t,  to) 

A22  22  A22 


$ (to)  “ I 
*22 


Section  37  SOLUTION  T = -TAj2T 


The  solution  of  the  homogeneous  quadratic  differential  equation 

f = 

with  implied  coupled  dynamics 


where 

X2  = TXj  (3) 

By  Eq.  (2) 

X2(t)  - 0 (4) 

or 

X^Ct)  = X^Cto)  (5) 

a constant. 

Using  Eq.  (5)  in  Eq.  (2) 

Xj  = A^2X2  = Aj2X2(to)  (6) 

integrating  Eq.  (6) 

Xj(t)  = X^(to)  + A^2X2('^o)(t-to)  (7) 

Using  Eq.  (3)  in  Eq.  (7) 


X^(t)  = [I  + A^2T(to)(t-to)jX^(to)  (8) 

The  psuedo  inverse  of  Eq.  (8)  is 

Xj(t)  = X*(to)[I  + Aj2T(to)(t-to)]"^  (9) 

using  Eq.  (9)  in  Eq.  (3) 

T(t)  = X2X*(t)  (10) 

= X2(to)X*(to)[I  + Aj2T(o)(t-to)]"^ 

T(t)  = T(to)[I  + A^2'r(o)  (t-to)]"^  (11) 


The  above  derivations  are  straight  forward,  however,  one  can  obtain  a solution 
via 

•-1  -1*  -1 
rp  * „ fj*  * TT  * 
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(12) 


or  using  Eq.  (1)  in  Eq.  (12) 

T-'  = A,, 

Integrating  Eq.  (13) 

T"^(t)  = T"^(to)  + Aj2(t-to) 
also  Eq.  (14)  can  be  written  as 

T"^t)  = [I  + Aj2T(to)(t-to)]T"^(to) 


Inverting  Eq.  (15)  one  obtains 

T(t)  = T(to)[I  + Aj2T(to)(t-to)] 
which  agrees  with  Eq.  (11). 
sting  ca 

= aV 


-1 


An  interesting  case  occurs  when  A^^  is  a rank-one  dyad  say 


(13) 


(14) 


(15) 


(16) 


12 

By  Eq.  (14) 


T(t)  = [T‘^(to)  + a"'^  Va  a] 


-1 


where 


a = t-to 

Using  the  Householder  matrix  inversion  limma  one  obtains 


T(t)  = T(to)  I - ^ o 


(17) 


(18) 


(19) 


(20) 


1 + 


.s 


i I 


? 


■) ! 


f ( 
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Section  38  SOLUTION  T = -TAj2T  + A2J.  This  section  obtains  a solution  of  the 
non-homogeneous  quadratic  equation 

T = - TAj2'r  ^2^  (1) 

T(to)  = To  (2) 

in  terms  of  a controlled  matrix  solution  to  Equation  (1)  with  a controlled 
initial  condition  which  may  simplify  the  computation.  Assume  we  can  select 
the  IC  such  that 

f = - T A-„T^  + A,„  (3) 

c c 12  c 12 


T (to)  = T 

c CO 


Note  that  the  simplified  dynamical  system  is 


0 A, 


^21  « 


X2  = T(t)Xj 

Since  Equation  (3)  is  non-homogeneous  it  can  have  dynamics  with 
T^(to)  = 0 

Note  that  if  we  try  the  inverse  dynamics 

•-1  -1  -1 
T ' = A^2  - ^21^ 

which  is  the  same  form  as  Equation  (1)  with  no  simplification. 
Let  the  desired  solution  be 


T(t)  = T^(t)  + N(t) 


where 


N(t)  = T(t)  - T^(t) 

N(to)  = T(to) 

The  derivative  of  Equation  (10  is 

• • • 

N = T - T 

c 

Using  Equation  (1)  and  Equation  (3)  in  Equation  (12) 
N = - TAj2T  - V^2Tc 


and  by  Equation  (9)  in  Equation  (13) 

N = - NCAj^T^)  + (-  T^Aj2)N  - NAj^N 


N = - NBjj  + B22N  - NBj2N 


N(to)  = T(to) 


«I1  ^2\ 


0 B. 


The  homogeneous  system  of  Equation  (15)  is  solved  in  Section  (26)  by  Equation 
(7)  and  is 

N(t)  = <t>  (t,to)[T(to)  + f % (T,to)A,,4>_  (T,to)dT]"^4>:^  (t,to)  (18) 

*^22  ■'to  *^11  ®22  ®11 


Bii  12  c Bjj 


<l>„  (to, to)  = I 
®11 


Vo  ■ Vl2Vo^’^’’^°^ 


♦n  (to, to)  = I 
®22 

Using  Equation  (18)  in  Equation  (9) 


T(t)  = T^(t)  -t  N(t) 


-1  -1  ' 

T(t)  = T (t)  + ♦ (t,to)[T(to)  + / ♦ (T,to)A,,4>_  (T,to)dT]  (t,to) 

®22  ■'to®ll  ®22  ®11 
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Section  39  SOLUTION  OF  T = + A22T  - TA^2‘^ 


This  section  obtains  a solution  of  the  homogeneous  bilinear  quadratic  equation 
T = -TA^j_  + A22T  - TA^2'*^ 


with  implied  dynamical  system 


All  Aj_2 


where 


X2  = TX^ 


The  dynamics  of  T is 

■-1  -1  ■ -1 
T = -T  TT 


- T-1  A22  + Aj2 


which  is  non-homogeneous  and  bi-linear.  The  solution  of  Equation  (5)  by 
section  33,  equation  (26)  is 

T"^(t)  - (t,  to)  [T(to) 

*11 

■"/toV  ”>*12»A,  <t.  to) 


N(t)  - $ (t,  to)[T(to)+  (T,  to)A3^2*B  to)dT]“^ 

°22  11  22  11 


where 

(t.  to)  - (A  + A T ) ♦ (t,  to) 

®11  11 

\ (t,  to)  = I 

®11 

■ ‘*22  - ■^o  *12>  “> 


(13) 

(14) 

(15) 


with 

$ (t,  to)  = I 

®22 

Using  equation  (12)  in  equation  (11) 
T(t)  = T^(t)  + N(t) 


(16) 


(17) 


where 


T (to)  = 0 
c 


(18) 


and  N(t)  is  given  by  equation  (12)  through  equation  (16). 

It  should  be  noted  that  in  ref  (47)  Lainiotis  uses  this  solution  technique  for 
the  special  case  of  the  Hamiltonian  system,  that  is 


with 


A 


12 


and 


(19) 


(20) 


(21) 
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Equation  (6)  is  called  the  process  dynamics  or  plant  and  equation  (8)  the 
measurement  vector.  Equation  (6)  is  to  be  thought  of  as  the  "true  process 
dynamics"  and  will  be  referred  to  as  the  "truth  model".  In  actual  practice 
one  usually  linearize  the  "true"  non-linear  process  dynamics;  in  which  case 
Equation  (6)  would  represent  the  linearized  equations  and  must  be  treated 
with  all  due  respect  for  linear  regions  in  which  they  are  valid. 


The  process  noise  and  the  measurement  noise  are  further  characterized  by 
their  variances.  Each  of  the  stochastic  noise  vectors  can  be  described  as 


“>j  = “>j  - ❖u 


a package-wise 


U = U - 1 

The  rank-one  dyad  of  Eq.  (12)  by  Eq.  (2)  can  be  written  as 
= Ul*>(l  = UPjj  = U 

where  in  "population  space"  or  the  larger  j-space 

!><! 


<11> 

is  the  rank-one  projector. 


Usir.g  Eq.  (13)  in  Eq.  (12) 


U = U - UPjj  = U(I  - Pj^) 


U = UP, 


where  the  orthogonal  complement  projector  is 


P^l  = I - 1 1 


which  is  rank  j-1 . 


A similar  relation  holds  for  the  measurement  vector 


V . = u + V . 
J V j 


with  zero  mean  v , that  is 


^ •ie  ^ ‘ie 

VI  = VPjjl  = 0 


The  variance  of  the  process  noise  is 


E[u  .-M  )(  u-  p)]=Eu.'^u 

J u-"'  J ^u^'  J 


1 
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or  matrix-wise 


Q(t,t)  = lim  UU  1 = KuyJ^u)  1 (21) 

j-«»  jmax  j = l ^ jmax 


The  limit  as  j max  goes  to  countably  infinite  will  be  be  assumed  in  all  of  the 


relation  and  not  written  down. 

The  serially  uncorrelated  assumption  is  used,  that  is 

E(u^^j^u)  = Q(t,T)6(t,T)  (22) 

and  similarly  for  the  measurement  noise 

E(:^j<^u)  = R(t,T)6)  (23) 

The  correlation  between  the  two  vectors  is  assumed  zero,  that  is 

UV"^  = E(u)'^v  = 0 (24) 

The  mean  of  Eq.  (7)  is 

and  the  mean  of  Eq.  (9)  is 

The  dynamics  of  the  error  vectors  is 
“ ’‘)j  ■ 

a package-wise 

X = AX  + BU  (28) 

The  observation  error  is 

">j  • “k  = ")j 

or  package-wise 

Z = HX  + V (30) 

Z = (H,I)  'Xj 

\v; 


at  time  T the  transpose  of  Eq.  (30)  is 

r - 

T 
H 

1 i 


Z^  = (x^,  v’^l 
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(31) 


r 


or 


assume  the  measurement  noise  is  independent  of  the  state  or 
E(i^<vl  = 0 = XV 


Dividing  Eq.  (32)  by  j max  one  obtains 

E[z^^z)  =]t^(t,T) 
zz 

or  Eq.  (33)  in  Eq.  (32) 


(t.t)  = (H,I) 


^-(t.t) 

0 


0 

R(t,T)6(t,T) 


or 


zz 


+ R(t,T)6(t,T) 


(32) 


(33) 

(34) 


(35) 


(36) 


Consider  next  a man-made  estimate  of  the  state  vector  with  dynamics  having 


the  structure 

x(t,t)>  = A x(t,t)>  + f(t))>  + C(t)>  (37) 

where  the  control  vector  c(t)^  (controlling  x(t,t^  to  follow  x(t)^  in 
some  sense  to  be  defined  later)  has  the  structure 

c(t)^  = W(t)z(t,t)^  (38) 

where  the  error  vector  z(t,t)  is  given  by 

z(t,t)^  = z(t)^  - z(t,t)^ 

z(L,t)>^  = Hx(t,t)>+  U>  (39) 

and  z(t,t)  is  the  estimate  of  the  measurement  vector 

z(t,t)^  = H x(t,t)]]>+  (40) 

Using  Eq.  (38)  in  Eq.  (37) 

x(t,t)^j  = Ax(t,t^j  + f(t)>  + BM(t^^  + W(t)  z(t,t)^^  (41) 


One  can  think  of  Eq.  (41)  as  a Curve  follower  or  positioning-serve  problem 
with  feedback  matrix  W(t)  and  when  z(t,t)  is  zero,  the  system  of  Eq.  (41) 
is  following  Eq.  (6) 

The  error  in  estimate  of  the  state  vector  is 

x(t,t^  = x(t^  - x(t,t^  (42) 
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I 


or  package-wise 

X(t,t)  = X(t)  - X(t,t) 

The  variance  of  the  state  estimate  is  given  by 

E(x><^(t,t))  = X(t,t)X^(t,t)  = P(t,t) 

j max 

The  minimization  criterion  is  to  find  a W(t)  which  minimizes  the  trace  of 
P(t,t)  which  is 


(43) 


(44) 


;(<^x>.)  = tr/  XX^  N = tr  P(t,t) 
* \j  max' 


and  then  to  find  the  dynamics  P(t)  for  the  optimal  W(t). 
The  error  dynamics  Eq.  (6)  minus  Eq.  (4)  is 

x(t,t]>^  = Ax(t,t)^  + Bu(t|>^  - W(t)  z(t,t;j^j 
Using  Eq.  (39)  in  Eq.  (46) 

x(t,t^j  = (A-WH)  x(t,t^j  + 
or  package-wise 

X(t,t)  = Aj.X(t,t)  + BU(t,t)  - WV(t) 

with 

Aj.  = A - WH 

The  homogeneous  system  has  dynamics 

\ 


and  the  fundamental  matrix  dynamics 

'e 


or 


j.  (A-WH)dt 

<t.,(t.to)  = 


with 


0j.(to,to)  = I 


The  solution  matrix  to  Eq.  (48)  in  fundamental  base  <J>j,  is 


X(t,to)  = 4'j,(t,to)X^(t,to) 


(45) 


(46) 


(47) 


(48) 


(49) 


(50) 


(51) 


(52) 


(53) 


(54) 
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f 


I 


The  time  derivative  yields. 

X(t,to)  = igX*’*  + 4>j,x'^ 
or  via  Eq.  (48)  and  Eq.  (52)  in  Eq.  (55) 

Ag,X  + BU  - WV  = A<l>gX'’’  + •t'gX'^ 
or 

BU  - WV  = 4>„X*’’ 

E 

or  the  apparent  velocity  is 

X*^  = tt>"\t,to)lBU-WV] 

The  solution  in  the  <t)„  bases  is 

E 

x'^(t,to)  = x‘*'(to,to)  + <l>'^(T,to)(BU-WV]dt 

to  ^ 


The  variance  by  Eq.  (44)  and  Eq.  (54)  is 

P(t)  = 4>j,(t,to)p‘*’(t)<t>g(t,to) 

where  the  variance  observed  in  the  <1>_  base  is 

E 

P*^(t,to)  = X*^(t,to)X*^^(t,to)  1 

j max 


The  dynamics  of  Eq.  (60)  is 

P = 4>g(t,to)P**’<Dg  + 4>j.p‘*’4>g  + 4)j,p‘*’4>g 
Using  Eq.  (51)  and  its  transpose  in  Eq.  (62) 


and  by  Eq.  (60)  in  Eq.  (63) 

• T ‘d)  T 

P = A^P  + PA^  + QgP'^cDg 

The  final  term  of  Eq.  (64)  must  next  be  evaluated  by  Eq.  (61) 

1 

j max 


(55) 

(56) 

(57) 

(58) 

(59) 

(60) 

(61) 

(62) 

(63) 

(64) 

(65) 


Consider  the  last  term  of  Eq.  (65)  by  Eq.  (59)  and  the  transpose  of  Eq.  (58) 


= *X*^(to,to)  + j 


to 


<f‘^(T,to)[BU(T)  - WV(T)]dXv 


x[U(t)B''’  - v'^(t)w'^(t)]4>‘^t,to) 
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(66) 


L 


(67) 


r 


F 


note  by  Eq.  (54)  and  (53) 

X*^(to,to)  = X(to,to) 
and  the  assumption  is  made 

E(x(to,to)^'  u(t)]  = E[x(to,to),>  '!^v(t)]  = 0 (68) 

' / 

or 

X(to,to)U^(t)  = 0 (69) 

and 

X(to,to)V^(t)  =0  (70) 

thus  Eq.  (66)  becomes 

X*’’(t)x‘'’^  = <l>‘^(T,to)[BU(T)  - WV(x)]  {U^(t)B^  - vV]4)‘'^(t,to)dx  (71) 

to 

The  parenthesis  terms  become 

[BU(X)  - wv(x)l  lU^(t)B^  - V^(t)W^  = BU(x)U^(t)B^  + WV(x)V^(t)W  (72) 

where  the  relation  of  Eq.  (24)  has  been  used.  Using  Eq.  (22)  and  Eq.  (23) 
in  and  Eq.  (72)  in  Eq.  (71) 

^ <|,'l(T,to)[BQ(x,t)6(x-t)B'’’  + WR(x,t)6(x,t)w'^]<l>'^x  (73) 

- to 

applying  the  6(T,t)  function  as  defined  in  appendix  (a)  Eq.  (17)  we  find 

x't’x'*’''’  = !54>‘\t,to)[BQ(t)B'’^  + WR(t)w'’']<t>g^(t,to)  (74) 

The  transpose  of  Eq.  (74)  is 

x't'x'*’’^  = ^<|)"\t,to)[BQB^  + WRW^]<t)"^(t,to)  (75) 

Using  Eq.  (74)  and  Eq.  (75)  in  Eq.  (65) 

P***  = <t>"\t,to)(BQB^  + WRW^]<|)"’'^(t,to)  (76) 

which  is  the  varience  dynamics  observed  by  an  observer  in  the  <!>„  space. 

Using  Eq.  (76)  in  Eq.  (64) 

P = AgP  + PaJ  + BQB^  + WRW^  (77) 

Eq.  (77)  is  valid  for  any  W(t)  selected.  An  optimal  W(t)  has  not  as  yet 
been  found.  Eq.  (77)  is  a matrix  Riccatti  and  implies  a dynamical  system 

= PYj  (78) 
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» 4 


1 


or 


or 


XX  = P = 


T T - 1 
P = Y Y (Y  Y ) 

2 r 1 r 


Since  P(t)  is  symmetric  so  is 

G, 

By  £q.  (211)  Section  025) 

Y. 


T T 

= Y Y*  = Y Y 
"21  2*1  *1*2 


1 


BQB^  + WRW^ 


^1  ; 


which  is  a Hamiltonian  system  with  solution  given  by  Eq.  (12)  Section  (36) 


(79) 


(80) 


(81) 


(82) 


as 


P(t)  = 4>E22(t,tO)  P(to)  + ' (D■\T,to)[BQB^  + WRW^]4>Ejj(T,to)]4>"Jj(t,to)  (83) 

. to 


where 


and 


• T 

<t)  — (h 

Ell  E Ell 


4)  = A <J> 

E22  E E22 


note  that  the  term  of  Eq.  (62) 

p = 4)„pW 
p EE 

by  Eq.  (76)  becomes 

P = BQB^  + WRW^ 

P 

The  homogeneous  part  of  Eq.  (64)  is 

P.  = A„P.  + P.  aI 
h Eh  h E 

with  solution  given  by  Eq.  (20)  Section  (30)  as 

Ph(t)  = 4)E(t,to)P(to,to)4>E(t,to) 

Thus  Eq.  (77)  can  be  viewed  as 

P = P.  + P 
h p 

Consider  now  Eq.  (77)  which  we  seek  to  minimize,  that  is 


P = A^P  + 


PA^  + BQB*  + WRW^ 


(84) 


(85) 


(86) 


(87) 


(88) 


(89) 


(90) 


(91) 
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where  by  Eq.  (49) 


A = A - WH 
e 


By  definition  of 


P = E’x(t,t)^<.  x(t,t)  :•  = lim  [XX  1 


jinax-»0  jmax 


when  trace  of  P is  minimized  with  respect  to  the  matrix  W,  then  trace  P 
is  also  a minimum.  The  gradient  matrice's  of  appendix  (F)  are  used  to 
find  optimal  W,  via 


3P  = 8 (A_P  + PAp  + WRW^) 

— ill  E> 

8W  8W 


(bob")  = 0 

.S..T 


By  appendix  (F)  summary 


[(A-WH)Pl  = - HP 
,T 


Transposing  Eq.  (96) 


_^(pA^)  = - HE 

8W^ 


The  quadratic  term  of  Eq.  (94)  has  gradient  given  by  the  transpose  of  Eq. 
(51)  appendix  (F)  as 


9(WRW^)  = 2RW^ 


Using  Eq.  (96),  Eq.  (97)  and  Eq.  (98)  in  Eq.  (24) 


8P  = - HP  - HP  + 2RW"  = 0 


HP  = RW" 


(100) 


T -1 
W = R HP 


(101) 


’a3WI»:».Tr'iSter,i:?ahii>ifS3 


and  transposing 

••  • . f. 

T -1 
W = PH^R  ^ 

which  is  the  optimal  gain. 


(102) 


Using  Eq.  (102)  in  Eq.  (91) 

P = (A-PH^R"^H)P  + P(A^-H^R"^HP)  + BQB^  + PH^R"^RR"^HP 


(103) 


P = AP  + PA^  - PH^R'^HP  + BQB^ 

Equation  (104)  is  a matrix  Riccatti  and  implies  a dynamical  system 
X2(t)  = P(t)X^(t) 
where  by  Eq.  (213)  Section  (25) 

:'x  -a'^  h'^r'^h^  , x \ 


which  is  a Hamiltonian  system.  If  X^  and  X2  are  nxn  matrices 
P(t)  = X^X'^ 

Expressing  the  inverse  as 

Xj'  = xJ(XjXj) 


P(t)  = X2X^(X^x]’)'^ 
The  product 


(104) 


(105) 


(106) 


(107) 


(108) 


(109) 


Vl  = ' 


P(t)  = (XjXj)'^ 


The  initial  condition  for  Eq.  (105)  for  any  Xj(t)  implies 


X2(to)  = P(to)Xj(to) 


Suppose 


(111) 


(112) 


Xj(to)  = I 


(113) 


X2(to)  = P(to) 


(114) 


(115) 


By  Eq.  (110),  for  any  Xj(t) 
X2(t)  = X''^(t) 


thus  Eq.  (106)  becomes 
.T 


‘1 


-A* 


BQB 


hV^h\  /Xj(t) 

x;^(t) 


with 


X2(to)  = P(to)Xj(to) 

Via  Eq.  (116)  and  Eq.  (82)  one  may  ask  what  is  T for 


=T 


'‘2I 


r 


(116) 


(117) 


(118) 


Summarizing  we  have  the  plant  stochastic  dynamics  given  by  Eq.  (6) 

x(t)'^  = + f(t)'^  + B(t)u(t)'^  (119) 

with  noisy  measurement  given  by  Eq.  (8) 

z(t)^  = H(t)  x(t)'^  + u(t)^  (120) 

The  optimal  estimate  of  the  states  has  dynamics  given  by  Eq.  (37),  Eq.  (38) 
and  Eq.  (102) 

^ A 

x(t,t)^  = Ax(t,tj>  + f(t)>  + P(t)H^R'^z(t,tfs  (121) 

where  by  Eq.  39 

z(t,t^  = z(t)')>  - z(t,t)^  (122) 

and 

z(t,t)^  = H(t)  x(t,tj>  (123) 

and  by  Eq.  (104) 


P = AP  + PA^  - PH^R"^H  + BQB^ 

One  must  initilize  Eq.  (121)  and  Eq.  (124)  with  an  x(o,o)^and  P(0) 


(124) 
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tA  in 


Section  42  CHANDRASEKHAR  TYPE  ALGORITHMS  FOR  SOLUTION  OF  OPTIMAL  KALMAN 
GAIN  MATRIX 

Recently  a number  of  papers  have  appeared  which  solve  the  matrix  Rlccattl 
differential  equation  which  describes  the  dynamics  of  the  variance  of  the 
state  estimation  via  a system  of  equations  called  the  Chandrasekhar  equations. 
Kallath  and  Lalnlotls  In  references  42  and  48  have  attempted  to  obtain  algorithms 
which  are  stable  and  require  less  computations  than  the  straight  forward 
solution  of  the  non-llnear  Rlccattl  equation.  There  Is  also  a close  relation 
between  the  bias  estimation  equation's  of  Frledland  and  the  Chandrasekhar  type 
equations  to  be  discussed  In  a later  section.  Both  methods  diagonalize  or 
decouple  equations. 

The  variance  Dynamics  by  Eq.  (124)  section  41  Is 


AP  + PA  - PH 


Fr'^p  + bqb’ 


and  by  Eq.  (77)  section  40  for  any  W 
P - AgP  + PA^  + BQB^  + WRW^ 

where  by  Eq.  (49)  section  41 


= A - WH 


and  by  Eq.  (102)  section  41  for  the  optimal  W 

T -1 
W = PH^R  ^ 

By  Eq.  (104)  section  41  for  the  optimal  W 


Fr'W  + bqb’’' 


AP  + PA  - PH  R 


P = AP  + PA^  - WRW^  + BQB^ 

Eq.  (6)  should  be  compared  with  Eq.  (2). 


By  Eq.  (105)  and  Eq.  (106)  section  41 


hV^ 


Kallath  derives  the  Chandrasekhar  type  algorithms  for  the  time  Invariant  case 
in  ref  42  and  Lalnlotls  in  ref  48  derives  the  time  varying  case.  This  section 
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will  merely  obtain  Che  constant  matrix  system  case  for  tutorial  purposes. 

The  derivative  of  Eq.  (6)  Is 

• T ’ T 'T 

P = AP  + PA  - WRW  - WRW 

where  the  assumption  on  Eq . (4)  Is 

■ ’ T -1 

W = PH  R 


The  transpose  of  Eq.  (10)  Is 

’T  -1 
W = R H P 

and  using  Eq.  (10)  and  Eq.  (11)  in  Eq.  (9) 

••  ■ ■ T ■ T T 

P = AP  + PA  - PH  W - VfHP 


P = (A-WH)P  + P(A^-hV) 


which  Is  homogeneous  and  non-quadratic,  with  solution  given  by  Eq.  (20)  sec  34 
p = <1.,^  (t)  p(o)  (t)  (14: 


The  Eq.  (13)  implies 


V = PV 
2 1 


-(A-WH) 


(A-WH)/  \ V 


and  the  fundamental  matrix  of  Eq.  (14) 
(t)  = e ■/to(A-WH)'’^dt 

AEii 

and 

4>.„  (t)  (A-WH)dt 

A£.22 

Note  that  Eq.  (16)  implies  adjoint 


V - V * 
2 1 


I 

J 


(20) 

(21) 

(22) 

(23) 

(24) 


The  constant  matrix  P(0)  can  be  expressed  as  a lower  triangular-diagonal-upper 
triangular  (LDU)  decomposition 

P(o)  = Lo  S Lo  (25) 

o 

where  S is  the  signature  matrix  of  P(0).  One  can  find  a theorem  on  page  91  of 
Perils  which  states  that  a real  synmetric  matrix  P(0)  of  rank  a is  congruent  to 
a matrix  S_ 


with 


Y(o,  o)  - 

Equations  (38)  through  Eq.  (40)  and  the  equations  given  by  Kailath  and  for 
some  systems  said  to  be  easier  to  solve. 
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APPENDIX  A 

THE  BINOMIAL  MATRICES.  RUTISHAUSER  MATRICES  AND  THEIR  INVERSES 


There  are  many  areas  of  stochastic  and  deterministic  dynamical  (con- 
tinuous and  discrete)  system  theory  where  one  is  lead  to  an  n-tuple  of 
elements  raised  to  powers.  One  of  the  most  fundamental  is  the  expansion 
of  the  exponentials 


ut 

e 


1 + (ut)^  + 


(1) 


where  u may  be  a real  scalar,  a complex  scalar,  a real  or  complex 
square  matrix.  The  vector  Taylor  series  expansion  of  a vector  along  a 
scalar  (time)  generated  trajectory,  the  characteristic  equation  as  a 
power-series  in  the  eigenvalues,  or  its  matrix  analog  via  the  Caley- 
Hamilton  condition,  or  the  fundamental  solution  of  a differential 
equation.  The  Fourier  series  frequency  domain  analysis  of  differential 
equations,  or  of  covariance  matrices  yielding  power  spectral  matrices 
through  the  Laplace,  Fourier,  etc.,  transforms.  The  complex  and  matrix 
case  will  not  be  strongly  developed  in  this  paper. 

Consider  sequence  of  reals  raised  to  powers 

^ = (u°,u^,u^,u^,- ••u^-*-d‘^'b  (2) 

and  the  change  of  variable 

u ■ z(x±y)  ■ zw  (3) 


where 


w * (x±y) 

then  the  jth  power  term  of  the  vector  of  equation  (2)  is 
=[z(x±y)]^  - z^[x±y]^ 


(4) 


(5) 


Bankier  in  his  paper  "Generalizations  of  Pascals  Triangle"  reference 
( 6 ) gives 


(x*y)^  - i (h  x‘  y 

1 A 


i j-i 

\r^ 


(6) 


i-0 


also  due  to  commutativity  of  x and  y 


724 


(7) 


(x+y) 


i=0 


where  the  binomial  coefficients  are  given  by  Morrison,  page  26  of  refer- 
ence (6 1) , as 


^i^  * i!(j-i)! 

where  j is  a non-negative  integer  and  i is  any  integer;  also 


<i)  = ■ Tpifn-! 

If  i>j , then  j-i<0  and  the  binomial  coefficient  is  taken  to  be  zero. 


Returning  now  to  the  transformation  of  equation  (3)  applied  to  the 
powers  in  equation  (1)  we  obtain  first  by  the  two  factor  products 


^ /•  0 z 1 d-1. 

<1  * (W  ,W,W  ,’"-w',*'*W  ) 


(10) 


<li  = wD(z^) 


(11) 


where  the  row  vector  w is  to  the  powers  and  the  diagonal  matrix  D(zj) 
is  given  in  equation  (10) . Thus  we  need  consider  the  w vector  and  for 
finite-finess  say  a 6x6  matrix  obtained  from  the  following  elements 


Wj^  = 1 = <5ce>j 


(12) 


where 


^ ,012  5. 

= (x  ,x  ,x  • • -x  ) 


(13) 


1 


pr-  T 


<$  ■ <X  T„c±y) 

Equation  (18)  can  also  be  written  as 

<w  = {l.±y,y^±y^.y'^,±y^)ri 


X 

x2 

x^ 

x^ 

5 

X 

1 

2x 

3x^ 

4x5 

5x^ 

0 

1 

3x 

6x2 

lOx 

0 

0 

1 

4x 

lOx 

0 

0 

0 

1 

5x 

0 

0 

0 

0 

1 

By  equation  (18)  for  positive  y unity  we  have  generating  equation  (7) 

wj  = (x+1)^  = J (j)  J (h  x^  (21) 


or  matrix-wise 

<w=<^xl  1 1 1 1 1 ='^B  =^T^(+1)  (22) 

0 1 2 3 4 5 

0 0 1 3 6 10 

0 0 0 1 4 10 

0 0 0 0 1 5 

_P  0 0 0 0 1_ 

which  are  the  well-used  binomial  coefficients.  We  will  call  the  B 
matrix  the  Binomial  matrix.  By  equation  (18)  for  the  negative  y case  we 
have 

wj  = (x-1)^  = I (j)(-l)^"^  x^  = I (j)(-l)^  x^‘^  (23) 


Just  as  we  observed  in  Section  (4)  equation  (164)  for  the  triangular 
Taylor  matrix  the  forward  and  backward  expansions  were  inverses  of  each 
other,  we  observe  here 


One  may  be  thought  of  as  a forward  propogation  by  unity  and  the 
other  as  a backward  (or  reverse  propogation  by  unity. 

Consider  the  translation  on  the  real  axis  of  equation  (21) 

w = X + 1 (26) 

which  power-wise  generates  the  transformation 

^ s ^xB  (27) 

and  the  inverse  translation 
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w = w - 1 


(28) 


which  generates  the  transformation 

<w'  = <wB(-l)  (29) 

Substituting  equation  (20)  in  equation  (28) 

w'  = (x+1)  - 1 = X (30) 

or  using  equation  (27)  in  equation  (29) 

<w'  = <xBB(-l)  = <x  (31) 

hence  in  general 

BB(-l)  = I (32) 

or 

B(-l)  = B'^  (33) 


which  generates  the  inverse  of  the  matrix  of  binomial  coefficients. 

Morrison  on  page  27  states  that  the  binomial  coefficients  (^)  may  be 
conveniently  displayed  using  the  well-known  Pascal's  triangle  in  which 
each  number  is  the  sum  of  the  two  above  it. 


1 1 


1 

3 

3 

1 

1 

4 

6 

4 

1 

1 

5 

10 

10 

5 

1 

1 

6 

15 

20 

15 

6 

1 

1 

7 

21 

35 

35 

21 

7 

1 

1 

8 

28 

56 

70 

56 

28 

8 

1 

1 

9 

36 

84 

126 

126 

1 84 

36 

9 

1 

1 10 

45 

120 

210 

252 

210 

120 

45 

10 

FIGURE  (1) 

FAMOUS  PASCAL  TRIANGLE  DISPLAY 
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1 


We  find  on  page  12  of  Miller's  book,  reference  (59]  an  improvement  of 
the  display  by  a table  array  shown  below  for  the  7x7  case  with  slight 
modifications 


0 

1 

2 

3 

4 

5 

6 

0 

1 

1 

1 

1 

1 

1 

1 

1 

0 

1 

2 

3 

4 

5 

6 

2 

0 

0 

1 

3 

6 

10 

15 

3 

0 

0 

0 

1 

4 

10 

20 

4 

0 

0 

0 

0 

1 

5 

10 

5 

0 

0 

0 

0 

0 

1 

6 

6 

0 

0 

0 

0 

0 

0 

1 

TABLE  I 

BINOMIAL  COEFFICIENTS  (^) 

1 

Miller  also  points  out  that  it  is  easily  seen  that  the  sum  of  any  two  inte- 
gers in  consecutive  rows  is  the  same  as  the  integer  to  the  right  of  the 
second  one  in  the  next  column.  Note  that  these  properties  stated  by 
Morrison  and  Miller  in  their  mnemonic  representations  are  quite  trivial 
with  respect  to  their  analytical  matrix  representations  and  related 
matrix  properties. 

If  we  designate  the  binomial  matrix  in  index  notation  where  i is  the 
row  and  j the  column  indexed  from  i,j=0,l,2, • • -d-l  we  have 


[bij]  = l(j)]  = f_JU I 


^00  ^01  ^02  • • ‘I 


(35) 


(S)  il)  (o)  il) 

{DiD 

il) 


(?) 

(0 


(V)' 


(i) 


(n 


(all). 


The  10  10  binomial  matrix  can  be  written  as  factorial  ratios  as 
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» > ' ■ ^ j ^ r • ' ■ !C,|i  ** 


and  the  triangle  of  Figure  (1)  the  ll^ll  case  is 


The  binomial  coefficients  (^)  are  also  known  in  terms  of  the  number  of 
combinations  of  j things  taken  i at  a time.  A few  other  areas  of  their 
occurrence  will  be  pointed  out.  In  section  (6)  we  find  a modified 
version  of  the  matrix  by  taking  successive  discrete  differences. 

Suppose  we  have  the  product  of  two  time  functions  and  take  the  deri- 
vative with  respect  to  time 

z(t)  = xy 

z = xy  + zy 

z = xy  + xy2  + xy 

z = xy  + 3xy  + 3xy  + xy  (38) 

etc,  or  package  wise 

(z(t)  ,z(t)  ,z(t)z')  = (x,x,x,x’) 


y 

y 

0 

0 


y y 
• •• 
2y  3y 


y 

0 


3y 

y 


(39) 


There  is  another  interesting  nested  representation  of  the  matrix  of 
Equation  (18)  we  note  that  for  the  d^d  case 


w°  = 1 = ^)XE^^y(^ 


(40) 


where 


= (x±y)^  = (l,x,x^)r  I)/ 

1:  “ I 


1 \ = <x  E B y>  (43) 


y dX3  (3^3)  3Xd 

xV 


= (x±y)^  = (l.x.x^x^)  ’ *■  / A 

_ / y 1 dX4X4  4xd 

.1  ' y2 

\y^ 


where  the  "select"  matrices  are 


E = I 
dX2  2X2 


(d-2)  x(2) 


etc,  the  linear-’ convolved  double-signed  binomial  coefficient  matrices 
are  designed  Be-  Packaging  the  equations  we  have 


<^w  = ^x  [Po.Fi.F2''^3' ‘^d-1^  ^ 


where 


T T 

E B E = E.  , B . , E 
c j-t-l  cj-^1 


The  pseudo  inverse  of  Equation  (47)  is  easily  obtained 


= ^j  + 1 \cj+l)  ^j+1 


where  for  example  the  inverse  of  the  5*5  element  is 


4 ^ 


A 


(55) 


■■■‘t  ■ ](l 


= (1 


3 r.  2 3.  r 

w = (l,x,x  ,x  ) 


1 


1 


etc.  For  example  consider  the  product 
T T T 

P/i  = Ec  B C Er  Ex  B , E* 

4 3 5 c5  5 4 c4  4 

dxd 

The  product 


eJ  E4  =r 

Sxd  dM 


i.! 

4)0j 


4x4 

.<4)0. 


and  using  Equation  (53)  in  Equation  (52) 

.T 


’‘Ih  - h 


dxd 


dX5 


0 0 
4 0 

0 3,6 


0 

0 

0 


0 

0 


3.4  0 
0 0 


(56) 


(57) 


(58) 


It  is  felt  (though  time  is  not  available  in  this  report)  that  the 
matrices  of  Equation  (58)  can  be  related  to  some  of  the  coefficients 
for  example  in  the  Legendre  and  Gram  polynomials.  Returning  now  to 
Equation  (18)  in  Equation  (11) 


<1  = <x  T^(±y)D(z^) 


(59) 
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where  the  upper  triangular  matrix  TuC±y)  of  Equation  (51)  is  spelled  out 
in  Equation  (18) . Equation  (59)  can  also  be  written  by  Equation  (18)  in 
Equation  (11) 

<u  = <(±y)  T^(x)D(z^)  (60) 

Returning  to  Equation  (24)  the  Tu(-l)  matrix  was  generated  by  the 
backward  propogation 

wj  = (x-l)j  (61) 


since  if  we  substitute  in  Equation  (7) 
y = -1 
we  obtain 

wj  = (x-l)j  = i (j)x^i  (-l)i 

i=0 ' ' 

Let  us  now  look  at  the  matrix  generated  by 

wj(-l)  = (1-x)^  = [(-l)(x-l)]j  = (-l)^(x-l)^ 
for  j=0,l,2* • -d-l ; we  have 

(l-x)°  = 1 = (-l)°(x-l)° 

(1-x)^  = • (-l)(x-l) 

. ► 

(1-x)^  = (-l)^(x-l)^ 

(l-x)*^"^  = (-D^-^x-l)*^-^ 
or 


(62) 


(63) 


(64) 
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[ (l-x)“,  (1-x)  , Cl-x)^,  • • • Cl-x)^  , • • • Cl-x)*^"^]  5 

= <Wb(-l)  =<%  ["1  , 1 


<Wb{-l)  =<Wb  I(-l) 


where  I(-l)  deisgnates  the  alternating  sign  matrix  of  ones.  By  Equation 
(63) 


wiC-l)  = (-I)-’  wJ 


By  Equation  (50)  and  Equation  (62)  in  Equation  (68) 

w^(-l)  = (-1)^  I 0.)  s^'^(-l)^ 

° i=0 

Using  Equation  (24)  in  Equation  (65)  we  find 

<Wb(-l)  = <(x  1 -1  1 -1  1 1+1 

0 1 -2  3 -4  5 -1 

0 0 1 -3  6 -10  H 

0 0 0 1 -4  10 

0 0 0 0 1 -5  0 

0 0 0 0 0 1 


= <x  T (-l)I(-l) 
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<Wb(-l)  =<X 

+1  *1  +1  +1 

+ 1 

+1 

(72) 

0 -1  -2  -3 

-4 

-5 

0 0+1+3 

+6 

+ 10 

0 0 0 -1 

-4 

-10 

0 0 0 0 

+1 

+5 

0 0 0 0 

0 

-1- 

By  Equation  (25)  we  see  that  Equation 

(22) 

is 

<w  = <xB  = <xT^(+l) 

(73) 

<?b  = = 

<xTJ-« 

(74) 

1 

1— • 

II 

00 

■'l(-l)  . <XT^(- 

1)I(- 

•1) 

(75) 

Westlake  on  page  (140)  calls  a matrix 

the 

Rutishauser 

R = fl  1 

1 1 ll 

(76) 

0 -1  -2  -3  -4 

0 0 13  6 

0 0 0 -1  -4 

_0  0 0 0 1, 

and  we  see  that  the  R matrix  is  the  matrix  of  Equation  (72)  and  Equation 
(70),  that  is 

B'^I(-l)  = R (77) 

From  Equation  (77)  it  is  obvious  what  the  inverse  of  the  Rutishauser 
matrix  is 

R’^  = r^(-l)B  = I(-1)B  (78) 

where 

r^(-l)  = I(-l)  (79) 
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observe  by  Equation  (80)  for  the  4^4  case 


1111 
0 12  3 

0 0 13 

0 0 0 1 


0 -1  -2 


0 1 


0 -1 


= I(-1)B  = R 


hence 


= I (82) 

or  the  Rutishauser  matrix  is  idempotent  index  2.  Todd  on  page  (240)  of 
reference  [86]  defines  the  condition  number  of  a non-singular  matrix  A 

c(A)  = ^ (83) 

where  X is  a root  of  largest  modulus  of  A and  u a root  of  least  modulus. 

Todd  states  that  the  eigenvalues  of  the  Rutishauser  matrix  are  either  1 
or  -1,  so  that 

c(R)  = 1 (84)  (84) 

Consider  the  Rutishauser  matrix  transpose 


1 -1 


I 


and  its  inverse  is 


T -1  T 
(R  R)  R 


-1  T T -1 
R ^ = R (RR  ) 

The  Grammian  of  Equation  (87)  is 


R R = 

1 

1 

1 

1 

1 

1 

2 

3 

4 

5 

1 

3 

6 

10 

15 

1 

4 

10 

20 

35 

1 

5 

15 

35 

60 

By  Equation  (35)  we  see  the  generalited  R matrix  is 


Hence  the  Grammian  matrix  of  Equation  (89)  involves  the  products  of  the 
binomial  coefficients.  Todd  on  page  (240)  of  his  book  gives  the  matrix 
product  elements  as 

R^R  = (91) 

where  by  Equation  (8) 

T 

The  inverse  of  R R is 

T -1  -1  -T 

(R  R)  ^ = R ^R  ‘ (93) 

by  Equation  (81) 

R = R'^  (94) 

and  transposing 
T -T 

R = R (95) 

hence  Equation  (94)  and  Equation  (95)  in  Equation  (93) 

(r\)'^  = RR^  (96) 

and  for  the  5^5  case 


T 

RR  = 


5 

-10 

10 

-5 

10 

30 

-35 

19 

10 

-35 

10 

-27 

-5 

19 

-27 

17 

1 

-4 

6 

-4 

Since  Euqation  (97)  is  also  a product  of  the  binomial  terms,  one  can 
obtain  the  general  expression  for  the  i-jth  element  which  corresponds  to 
Equation  (91). 


✓ • 


1 


1 


w 


We  also  have 

RR^  - RR^I  = RR^(RR"’)  (98) 

RR^  = R(R^R)R"^  (99) 

and  by  Equation  (96)  in  Equation  (99) 

RR^  = R(RR^)"^  R'^  (100) 

T T -1  . . 

which  implies  that  the  eigenvalues  of  RR  and  (RR  ) coincide. 

By  Equation  (63)  we  see  that  any  function  sequence  of  the  form 

gj(t)  = [i-fi(t)]^  = i^(j)  (-1)^  fi(t)  (101) 

or  packagewise 

<g(t)  = <f(t)R  (102) 


generates  the  Rutishauser  matrix.  Westlake  on  page  (140)  gives  the  left 
eigenvector  corresponding  to  the  three  unity  eigenvalues  for  R as 


= 1,  i = 1,2,3 


as 


<S,u  .( 


6X2*  4x4 , 3X2' 2x^ , Xj , x^ , Xg 


or  factor-wise 

^5)u  = (X2,X^,X5) 


6 3 10  0 

-4-2010 
0 0 0 0 1 


and  the  two  eigenvectors  corresponding  to 


(103) 


(104) 
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■lUiiiliiiilii 


I 


X. 

: 


•1  j=4.5 


as 


^5)u  — (0, 2x^-Xg , 2x^-Xg ,x^ ,Xg) 


and  factor-wise 


^)u  = (x^.Xg)  To  2 2 1 ol 

^ [o  -1  -1  0 ij 


or  packagewise. 

=4 


I 

0 

3x3 

3x2 

0 

-I 

2x3 

2X2 

= UA  = UR 


where 


/ 


3x3 


(105) 


(106) 


(107) 


(108) 


\ ° "2x2 

also  the  right  eigenvectors  are  given  by  Westlake  as  for  Xi=l  i=l,2,3 


(5>  = 


^1 

^2 

ys-yi 

-2/5 

/s 


10  0 
0 10 
0-11 
0 0-2 

0 0 1 


(109) 


.1 


1; 

1: 
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1 


0 


(110) 


and  for  Xj«-1  j=4,5 

v(5)>  = 

j 


yi 

1 


and  packagewise 


I 

0 

3x3 

3x2 

0 

I 

2x3 

2X2 

7<-%) 

^4 

0 


V = RV 


-2 

0 

0 

0 


^4 


Returning  now  to  Equation  ( 4 ) 
w = X ± y ^ 

we  shall  recast  three  cases  corresponding  to 
w = X + y 
w = X - y 

for  yj^O,  normalize  Equation  (113) 

« = w = 1 = X +1 

y y y y 

and  Equation  (114) 


w^ 


w,  1 

by  y 


and  for  Xj>f0,  normalize  Equation  (114) 


(111) 

(112) 

(113) 

(114) 

(115) 

(116) 
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S ^ ' 


- 


m 


' ' 


T 


The  powers  of  Equation  (115)  generate 
<"y  =■  <V 

and  the  powers  of  Equation  (116)  generate 

<rby  ■ <V'-« 

and  the  powers  of  Equation  (117)  generate 

<“x 

we  also  have 

4y  = <xD'^y") 

<(Wy  = <wD‘^(y^) 

<V  = 

Using  Equations  (121)  and  (122)  in  Equation  (118) 

<(w  = <xD-l(y")BD(y") 
corresponding  to 
w = X + y 


(117) 

(118) 

(119) 

(120) 

(121) 

(122) 

(123) 

(124) 

(125) 

(126) 


Wfa  = X - y 


(128) 


and 

<Wb(-l)  = <xD‘^(y")RD(y'') 
corresponding  to 

WjjC-l)  = X - y 

where 


D(y”)  = 


.d-1 


Reconsider  Equation  (6)  for  x“y=l,  we  have 


(129) 


(130) 


(131) 


I 


(132)  ' 


or  as  an  innerproduct 
2 


(133) 


The  binomial  matrix  of  Equation  (34)  has  the  property  using  Equation 
(13o) 


(134) 


(135) 


r ^ 


Now  reconsider  Equation  (7)  for 


X = 

y = 


!) 


then 


0= 


(136) 


i=0 

and  as  an  inner-product 


The  Rutishauser  matrix  of  Equation  (76)  thus  has  the  property 


R = B'^I(-l)  = <^)  0 


(137) 


(138) 


y-Binomial  Matrix  and  its  Inverse.  For  lack  of  better  terminology 
the  y-binomial  matrix  will  be  defined  as  the  matrix  of  Equation  (18)  for 
plus  sign  case,  that  is  the  real -axis  translation 


w = X + y 


(139) 


induces  the  monomial-base  transformation 

= <5cB(y)  (140) 

where  B(y)  is  the  binomial  matrix  carrying  along  the  variable  y;  by 
Equation  (35) 


1 

•j 


j 


B(y)  = 
dxd 


\i)  (^o>  {y  (^)> 

» (i)^  (3^ 


/d-l\  d-1 

(oji- 

/d-l\  d-2 

Vl)" 


or  for  the  6x6  case 


B(y)  = 1 


2 3 4 

1 2y  3y  4y  5y 

2 3 

0 1 3y  6y  lOy 


0 0 


0 0 


0 0 


4y  lOy^ 


{tl)^ 


If  we  make  the  additional  variable  change  on  the  real  axis 


w - y 


at  the  package-of -power  level  by  Equation  (18)  for  negative  sign 





adding  Equation  (139)  to  Equation  (143) 
w''-x  + y-  y = x 
and  package-wise 

Substituting  Equation  (140)  in  Equation  (144) 
= <^B(y)  B(-y)  - ^xl 


or 


or 


B(y)  B(-y)  = I 


B(-y)  = B‘^(y) 


where  for  example 
B"^(y) 


. 

2 

3 

4 

5 

1 

-y 

y 

-y 

y 

-y 

0 

1 

-2y 

3y^ 

-4y^ 

0 

0 

1 

-3y 

6y^ 

1 

H 

O 

0 

0 

0 

1 

-4y 

2 

lOy 

0 

0 

0 

0 

1 

-5y 

_0 

0 

0 

0 

0 

1 

If  we  make  the  translation  by  Equation  (139) 
r=w+q=x+  (y+q) 
or  package  wise 

^r  - ^B(q)  ■ ^B(y+q) 
using  Equation  (140)  in  Equat  ion  (152) 
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adding  Equation  (139)  to  Equation  (143) 


w'=x  + y-  y = x 
and  package-wise 
= <xi 

Substituting  Equation  (140)  in  Equation  (144) 
= ^xB(y)  B(-y)  = <^xl 


(145) 


(146) 


(147) 


B(y)  B(-y)  = I 


(148) 


B(-y)  = B"^(y) 


(149) 


where  for  example 


B (y)  = 1 -y 

0 1 


■y 

y 

-y 

y 

-y 

1 

-2y 

3y^ 

-4y^ 

1 

5y 

0 

1 

-3y 

6y^ 

-lOy' 

0 

0 

0 

1 

-5y 

c 

0 

0 

0 

1 

If  we  make  the  translation  by  Equation  (139) 
r=w+q=x+  (y+q) 
or  package-wise 

^ = ^wB(q)  = ^xB(y+q) 
using  Equation  (140)  in  Equation  (152) 


(151) 


(152) 


J 


Appendix  (B)  BACKWARD  AND  FORWARD  FRACTORIAL  FUNCTION  MATRICES.  STERLING  AND 
ASSOCIATED  MATRICES  AND  RELATIONS  TO  RUTISHAUSER  AND  BINOMIAL 
MATRICES 

Consider  the  (i  - element  of  the  Binomial  matrix  of  Eq.  (35)  sec  (A) 

J « i! (1) 

1 il(j-i)!  ^ 

and  the  quotient  of  Eq.  (1) 

■1!  , i(1-l)(i-2)  ...  (j-H-Dd-l)  ...  2 ..1 

(j-1).'  (j-l)(j-l-l)  ...  2 .1 

and  after  cancelling  common  terms 

= j(j-l)(j-2)  ...  (j-1  + 1)  (3) 

Equation  (3)  is  a special  case  of  the  backward-factorial  function  of  order  i 
written  as 

j^^^  = j(j-l)(j-2)  ...  (j-1  + 1)  (4) 

when  j is  less  than  1,  then  Eq.  (4)  is  equal  to  zero  as  can  be  seen  by  entering 
the  values  in  Eq.  (4). 

Brand  in  his  paper  "Binomial  Expansion  in  Factorial  Powers"  states  that  in  the 
calculus  of  finite  differences  the  factorial  powers  x(n)  plays  a role  analogous 
to  that  of  xn  in  the  differential  calculus.  He  states  that  there  is  not  only  a 
wide  diversity  of  notations  for  the  factorial  powers  but  also  a disagreement  as 
to  its  definition  when  the  index  is  negative.  Brand  says  that  moreover  the 
notation  (ax  + b) (n)  is  sometimes  used  in  a way  which  may  be  a fertile  source  of 
error  and  that  the  way  out  of  this  confusion  is  to  use  a single  definition 
applicable  to  all  real  values  of  the  index.  He  states  further  that  the  alleged 
factorial  powers  (ax  + b) (n)  are  readily  converted  to  genuine  factorial  powers. 

When  n is  a positive  integer  the  backward  factorial  function  (x  a real  variable) 
of  order  n is  defined  as  the  product 

x^"^  - x(x-l) (x-2) (x-3)  ...  (x-n  + 1)  (5) 

Thus  x^**^  is  the  product  of  n terms  starting  from  x and  counting  down  by  unity, 
for  example  if  n = 3 

x^^^  » x(x-l) (x-2)  (6) 

Also  the  forward  factorial  function  of  order  n is  defined  as 

x^'^  * x(x+l)  (x+2)  (x+3)  ...  (x+n-l)  (7) 

and  for  the  example  n = 3 

= x(x¥l)(x¥2)  (8) 


rmsmassisf 


(9) 


Note  by  Eq.  (4)  that 

(j) 


- j(j-l)(j-2)  ...  3-2-1  = j! 

Returning  to  the  definition  of  Eq.  (5) , consider 


= x(x-l)(x-2)  ...  (x-m  + 1) 


(10) 


and 


( ) 


(x-m) ' * (x-m) (x-m-1)  . . . (x-m-n+1) 


(11) 


and  the  product  of  Eq.  (10)  and  Eq.  (11)  yeilds 


(m)  , ^ (n)  (mfn)  (n)  , . (m) 

X (x-m)  = X = X (x-n) 


(12) 


Miller,  ref  (59)  says  that  the  definition  of  Eq.  (5)  when  n is  a negative 
integer  stems  from  the  same  type  of  reasoning  used  in  defining  negative 
exponents  in  elementary  algebra,  that  there  it  is  easily  proved  that  for  n 
and  m positive  integers 


n m n+m 

XX  = X 


(13) 


and  if  this  rule  of  exponents  is  to  hold  for  non-positive  integers  one  is  in- 
evitably led  to  the  definitions 


0 1 
X = 1 


and 


-n 


l/j 


Brand  considers  Eq.  (12)  to  obtain  a definition  for  all  real  values  of  m and  n 
when  m = 0,  Eq.  (12)  becomes 


(0)  (n)  (n) 

X X “ x'  ' 


(14) 


hence  if  x 0 

..(0)  - 


= 1 


(15) 


When  m = -n  Eq.  (12)  yields 


(0)  (-n)  (n) 

X = X (x+n) 


(16) 


or 


.(-n)  = 


(x+n) 


(n) 


(x+n)(x4n-l)  ...  (x+2) (xfl) 


(17) 
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or 


j<  > (d- 


-1  ■ " ^ '>j 

Solving  for  the  vector  of  binomial  coefficients 


(26) 


I (I  I/O' 

(ii  = 


1/1! 


1/2! 


i j 


i(0) 

,(1) 

(2) 


(27) 


(d-l)l  J 


|j<d-l) 


The  vector  of  Eq.  (26)  is  seen  to  be  the  j 
Eq.  (35)  sec  (A),  hence  packaging  Eq.  (27) 


th 


column  of  the  Binomial  matrix  of 


B = * 


,-l  c(  ) 


(28) 


where  the 

matrix  of 

backward 

factorial 

functions 

s<  > 

is  given  by  (definition 

herein  used  is  restricted  to 

j = X integers) 

r 

1 

\ 

o(o) 

3^(0) 

.,(0) 

^ • 

^(0) 

1 • • • 

...  (d-l)^°^ 

1 

! 

0 

l(l) 

2(1) 

id) 

j(l) 

(d-1)^^^ 

: s(> 

0 

2(2) 

i(2) 

j(2) 

(29) 

dxd 

0 

i(3) 

• 

• 

;(i) 

• 

• 

1 

0 

;(j) 

L 

(d-l)(‘^-l> 

or 

(30) 

i.  j = 0, 

1,  2 ... 

(d-1) 

For  example,  the  5x5  case  is 
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or 


1 1 

1 

1 1 

s<  ) 

0 1 

2 

3 4 

> 

0 0 

2 

6 12 

(31) 

5x5 

0 0 

0 

6 24 

0 0 

0 

0 24 

forward  factorial 

function  matrix 

is  defined 

as 

- 

pi 

(32) 

r 

^{0) 

2^05  . 

(d-l)^°^ 

iCil 

2cn 

(d-1)'^’-^  ! 

sU 

2Z2^ 

,12) 

i 

(33) 

i... 

• 1 

J 

• 1 

(d-1)^^"^^  ! 

for  the  5x5  case 

r 

1 1 

1 

1 1 

sU 

: 

0 1 

0 0 

2 

6 

3 4 

12  20 

(34) 

5x5 

0 0 

0 

60  120 

0 0 

0 

0 840 

By  Eq.  (28) 

^ i B 

hence  we  can  now  easily  invert  the  backward  factorial  function  matrix 

where  the  Inverse  binomial  matrix  is  given  by  Eq.  (77)  of  appendix  (A) 
in  terms  of  the  Rutishauser  matrix 


B 


hence 


-1 


RI(-l) 


^5^  * Rl(-l) 

Morrison  ref  (5)  gives  the  generalization  of  Eq.  (24)  as 


(35) 


(36) 


(37) 


(38) 
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r7  '7!  ■ 


I 


ip-  0 


i = 0 


i<  0 


for  X any  real  number  and  i any  interger. 

We  next  seek  a relation  between  powers  of  x and  the  backward  factorial  functions 
of  Eq.  (5),  which  yields  for  the  5x5  case 


fl  0 


0 -1 


0 0 
0 0 


0 2-3 

0 -6  11 


> = x> 
b 


The  matrix  S,  is  called  the  Stirling  Matrix  of  The  First  Kind  and  in  this  report 
will  be  further  classified  as  the  Backward  Stirling  Matrix. 

If  we  partition  Eq.  (40)  as 

/x(0\  "°Zs. 


hence  the  row  vectors  of  Eq.  (40)  are  the  elements  of  The  eleven  by  eleven 

case  is  given  in  Morrison  ref  (61).  Note  by  Eq.  (42) 

x>  . (43) 


If  we  now  let  x “ j (the  Integers)  we  have  the  connection 


759 


or  by  Eq.  (29) 


Using  Eq.  (38)  in  Eq.  (52) 

- S':! 

or 

and  the  inverse  of  the  S,  matrix  is 


and  by  Eq.  (37) 

s~^  - Ri(-i) 

- Cg  RF"^I(-1) 


(52) 

(53) 

(54) 

(55) 

(56) 
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The  inverse  of  the  Backward  Stirling  Matrix  also  called  Stirling  Matrix  of  the 
First  Kind  is  called  the  Stirling  Matrix  of  the  Second  Kind  and  its  matrix 
factors  are  given  by  Eq.  (56).  Morrison  on  page  25  of  ref  (6l)  gives  the  11x11 
table  to  construct  the  matrix,  (he  does  not  obtain  the  matrix  factors  of 
Eq.  (56),  they  are  derived  here  for  the  first  time  to  my  knowledge).  The 
first  five  are 


(57) 


we  see  also  by  Eq.  (54) 


the  inverse  of  the  C matrix  can  be  obtained  as 

£i 


(58) 


In  a similar  manner  by  Eq.  (7)  we  can  formulate  the  forward  factorial  vector 
in  terms  of  the  monomial  base  for  the  5x5  case  as 


(59) 


(60) 


If  now  we  let  x = j we  have  the  connection 


(61) 


and  the  vector  of  integer  powers  is 

Z.j  = (1.  j.  ...  ...  (62) 

for 


j =0,  1,  2,  ...  d-1 

The  matrix  of  Eq.  (59)  and  Eq.  (60)  is  called  the  Forward  Sterling  Matrix  in 
this  report  (it  is  the  transpose  of  the  matrix  of  Morrison  p.  85  which  he 
calls  the  associate  Sterling  matrix  of  the  First  Kind. 

The  5x5  case  is 


762 


Ti 

0 

0 

0 

0 

» 

i 

i 0 

1 

0 

0 

0 1 

1 

- 1 0 

1 

1 

0 

0 i 

1 

(63) 

L 

0 

2 

3 

1 

0 i 

5x5 

LO 

6 

11 

6 

ij 

t 

If  we  package  Eq. 

(61) 

into 

a row 

of  column  vectors  for  j =■  0, 

1,  2 ...  we  obtain 

the  left 

hand  side  matrix  by  Eq. 

(33) 

as 

^ * Sj  Cg  (64) 


in  analogy  with  Eq.  (50),  or 

Sf  = C"''^  (65) 

By  Eq.  (52)  in  Eq,  (64) 

^ = S.  S'^  ^ (66) 

t D 


which  connects  the  forward  and  backward  factorial  function  matrices. 


The  inverse  of  the  forward  Sterling  matrix  of  Eq.  (65)  is 


(67) 


The  inverse  of  the  forward  Sterling  matrix  is  called  by  Morrison  the  associate 
Sterling  matrix  of  the  Second  Kind  (he  uses  the  transpose  matrix);  the  5x5 
elements  are 


5x5 


10  0 
0 10 
0-11 
0 1-3 

|0  -1  7 


0 

0 

0 

1 

-6 


— I 

0, 

0 

0 

0 

1 


(68) 


Many  other  relations  between  the  Binomial,  Rutlshauser  and  these  matrices  can 
be  obtained  but  are  not  pursued  further  here.  One  obvious  one  is  by  Eq.  (38) 


R ♦ I(-l) 


(69) 


and  by  Eq.  (81)  appendix  (A) 


(70) 


or 

s^  ^ ^ I(-l)  [s^  t I(-1) 

and 


(71) 
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(72) 


^ I(-1) 


which  provides  the  factors  of  the  inverse  of  the  backward  factorial  function 
matrix. 

If  we  have  a polynomial  in  the  monomial  base 

f(x)  = ZxaA  (7 

by  Eq.  (41) 


f(x)  = Z.x^  a^ 

D 


f (x)  » < x^  ^ b > 


where 


b>  = S a> 

D 


Pol^^mlals  in  the  factorial  function  base  are  called  Newton  Polynomials  and 

will  be  referred  to  as  the  Backward  Newton  Base,  connected  to  the  monomial 
base  via  Eq.  (4). 


APPENDIX  C 


FORMULAS  FOR  THE  SUM  OF  THE  dth  POWERS  OF  THE  FIRST  N INTEGERS 


L.  S.  Levy  in  Reference  (52)  shows  that  for  each  positive  integer  d 
there  is  exactly  one  poljmomial  S(j(N)  in  N such  that 


S,(N)  = 1 + 2^^  + 3^  + •••  n'^ 
d 


whenever  N is  a positive  integer. 

Equation  (1)  can  be  written  as  an  irner  product 

S^(N)  = <0*^  ^ 

where  the  row-vector  is 

= (1,  2^^,  3^,  4^^,  •••  N*^) 

and  the  column  vector  is  an  N dimensional  vector  of  ones 


s^(„)  = I . 5-  = I (Ntl) 


2 

N N N N 

S2(N)  = + — + f”  = '6  (2N+1) 
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(1) 


(2) 


(3) 


(4) 


The  following  formulas  are  given  in  Reference  (‘<2)  for  d=l,2'*’7.  Powers 
beyond  this  can  be  obtained  from  the  papers. 


(5) 


(6) 

(7) 


J 


^4^^^  ^ ^ 3N^+3N-1)  (8) 


N^  SN*^  N^  N^  N^ 


S^iU)  = - ^ + ^ + ^ + ^ = ^ (N+1)2  (2nWi) 


N N^  N^  N^  n"^  1 4 3 

SgCN)  =1^2"6"'^~'^“'^?“^47  N(N+1)(2N+1)(3N  +6N  -3N+1)  (10) 

q (N)  = — 7^*^  7N^  N^ 

SyCN)  24  ■*■  1^  "^2  "^8 


|j^  (N+1)^  (3n‘^+6N^-N^-4N+2) 


The  span  width  in  points 

W =2M  + 1 = N + 1 
P 

2M  = N 


W - 1 = N 
P 


The  summations  of  Equations  (5-11)  in  terms  of  Wp 

S (N)  = <(lc^  = S (N)  = ^ (N+1)  = i (W  -1)  (W  ) 


S2(N)  = <(lP^J>=  I (N+1)  (2N+1) 


(W  -1) 


s^Cn)  = ■ '■  * 


(W  ) (2W  -1) 
P P 


(18) 


. m2  o (W  -1)  o 

SaCN)  =<ic>=fKN+i)  =-V— ”p 

SuCN)  = <5-0*^  = 1^  (N+1)  (2N+1)  (3N^+3N-1) 

W 2 

= (W  -1)  (2W  -1)  (3W  -3W  -1) 

30  p p P P 

The  summations  of  Equations (5-11)  in  terms  of  M 
S^(M)  = <^lc^  = I (M+1) 

■m 

• ? [“p-O 

S^dl)  = <^lc^y  = I (M+l)  (2M+1) 

(w  -1)  pw  +1-1 

2 J^ 


w -ir  w 
_L 

4 

w -1  r w +1* 

_E_ 

4 


6(2)  I 2 J p 

Wp  (»p-i) 

2 


S3(M)  = <Lc^(i^  = (M+1) 


(W  -1)  (W  +1)^  (W  -1)(W  +1)^ 
P P - = P P 


(19) 


(20) 


(21) 


(22) 


16 
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Appendix  (D)  INTEGRATION  BY  PARTS  OFjt^e^^dt 

This  appendix  derives  some  classical  integrals  by  integration  by  parts. 
Consider  the  integral  where  n=o,  1,  2 . . . d-1, 


Jt”  6 ^^dt  = Ju  dv 


where 


du  = nt*'  ^dt 


dv  = 6 ^^dt 


V =/e“dt  = 


The  integration  by  parts  comes  from 
d(uv)  = du  V + u dv 


d(uv)  =uv=^duv+  Ju 


udv  = uv  - I vdu 


Hence  using  Eq.  (2)  through  Eq.  (4)  in  Eq.  (8) 
/t"e""dt  = t^el"  - n 


a a 


In  the  same  manner  one  can  shov 


If  we  apply  the  end  points  of  integration  the  definite  integrals  Eq.  (10) 
becomes 


f t“  e-'^^dt  = 
^ ti 


. t2  ,t2 

tnf  t 

a tl  “'tl 


As  the  first  case  considers  the  interval  (o,  a>) , then  Eq.  (11)  becomes  for  n = 


The  occurance  of  oo/oo  necessitates  the  use  of  L.  Hospitals'  rule  taken  from 
Schaum's  Series  "Differential  and  Integral  Calculus"  page  114  reference  (78) 

L.  Hospitals'  Rule:  If  b = + <»,  or  if  f (x)  and  g(x)  are  differentiable 
and  g(x)  = 0 for  all  x or  some  interval  1 x 1 > M,  if 

lim  f(x)  = 00 
x^b 


and 


lim  g(x)  = 00 
x-»b 


then  when 


-1. 


lim  / f (x)\  exists 

[TTJ 

or  is  infinite 

lim  /f(x)\  = lim/f^(x) 
x^b  \g(x)J  x^b  1 

Vg  (x), 


as  an  example 


lim/  X \ = lim/2x\  = lim/2\=  0 


x^\  e 


x-«»  \ 0 


x->o  \ e ‘ 


applying  Eq.  (15)  to  Eq.  (IJj)  we  see 


(15) 


(16) 


r * 


th 


and  likewise  by  repeated  applications;  the  n degree  term  Eq.  (11)  is 


^ng-at 


_ , . . np  -at  . 

= lin  t = 0, 

0 t^ 


(18) 


which  is  a repitition  of  the  example  of  Eq.  (16). 
Using  Eq.  (17)  in  Eq.  (13) 

Continuing  in  this  recursive  manner  one  can  show  that 


(19) 


_0O 

r ..n_-at 
/ t e = n! 

_ n 


(20) 


n+1 


Clearly  if  one  replaces  the  interval  of  integration  by  (0,  - »)  the  term 
are  bounded  for 


/-CD 

e-at 

0 


dt  = - 6 = “ 


(21) 


However  the  positive  exponential  power  of  Eq.  (9)  is  finite  in  the  interval 
(0,  - <») , for  n = 0 we  have 


P 


atj^  _«at 


e ^’'dt  =e 


-00 

= -1 
0 a 


(22) 


th 


and  for  the  n term 


0 


dt  = n! (-1) 


n+1 


(23) 


n+1 


Contrast  this  alternating  sign  result  with  Eq.  (20). 
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Appendix  (E)  BASE  AND  METRIC  RELATIONS 

Consider  two  bases  connected  via  Eq.  (1)  where  T is  known 

<t  = 

and  suppose  one  performs  a G-S  process  as  the<^  vectors  such  that  is 
known 


where 


a>@<S  = I 


\i ' ^>©<^  = "tt  = 


We  desire  to  find  the  G-S  connection  matrix  B where 

s 


under  the  constraint 


s>©<s  = I = o>©<a 
By  Eq.  (2)  and  Eq.  (5)  in  Eq.  (6) 

= Bji>©<iB^ 


B^M  B = B^M  B 
s tt  s o IT  a 

a "quadratic  algebraic  equation"  in  B^  the  unknown. 

Using  Eq.  (4)  in  Eq.  (8) 

T T T 

B T M TB  = B M B 
s It  s o IT  o 

a solution  is  given  by 


where 


B = t'^B 
s a 


B^  = b^t"^ 

s a 


and  substituting  Eq.  (10)  and  Eq.  (11)  in  Eq.  (9) 

T -T  T -2  T 

B T T M TT  B„  = B^M^^B^ 
a IT  0 0 IT  o 

However  Eq.  (10)  is  not  a unique  solution  for  example  another  solution 

B = LT'^B„ 
s 0 


and 


T T -T  T 

B = B T L 
s a 

and  using  Eq.  (13)  and  (14)  in  Eq.  (9) 

T -T  T T -2  T 

B T L T M TLT  B = B M B 
o TT  a o XX  a 

for  all  L and  T such  that 
TLT'^  = I 


(14) 


(15) 


(16) 


Note  that  Eq.  (9)  is  a special  case  of  the  steady-state  matrix  Riccati 
differential  Equation  and  many  studies  and  papers  have  been  published  along 
this  line  but  will  not  be  pursued  further  here. 

Note  that  we  have  the  further  relation  using  Eq.  (1)  in  Eq.  (5) 

C'i  = tTB  (17) 

and 

I = s>0<s  = BVt>0<TTB^  (18) 

The  metric  factors  were  derived  in  Eq.  (348)  of  Section  (1)  as 

.T,-l 


or  Eq.  (19)  in  Eq.  (8) 


and 


T T T -1 
I = B T (B  B ) TB 
s a a s 


-T  -1  T- 

= B B = (B  B ) 
tt  s s s s 


T T -1 


and  inverting 

-1  T -1  T 

mJ  = B P = T B B 
tt  s s a o 

Equating  factors  we  find 

B = t'^B 
s a 


(19) 


(20) 


(21) 


(22) 


(23) 


but  we  know  from  standard  matrix  theorems  that  triangular  factors  (for  full 
rank  matrices)  are  only  unique  when  they  are  in  unit  triangular  factors  with 
a diagonal  matrix. 


As  an  example  of  a third  solution 

B = T'^B  T 
s a 

T T T -1 
B = T B T 
s a 


(24) 

(25) 
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and  one  obtains 


T^(b3m^  B„)T  = B^M  B 
O TT  O TI  O 


(26) 


hence  for  all  T that  realize  the  congruent  automorph  relation  of  Eq.  (26) 
one  has  Eq.  (24)  as  a solution  to  £q.  (9). 


APPENDIX  F 

GRADIENTS  OF  TRACES  OF  MATRIX  FUNCTIONS 
Some  well  known  trace  properties  of  matricies  are 

tr  M = tr  (1) 

and  the  cyclic  property  of  products.  For  example 

trR  ST=trTRS=trSTR  (2) 

pxm  mxjl  )ixp  ixp  pxro  mxA  ^xp  pxm 

A.  LINEAR  CASES.  Consider  the  matrix  function  of  the  rectangular 
iratrix  X of  size  pxm,  that  is 


A X B 


with  trace 


tr  ^ = "tr  AXB  O^ote  this  Z is  not  the  matrix  size.) 


the  mxm  matrix  L-- 


Lg  = B A X 
.nxn,  P^m 


with  trace 


tr  I12  = tr  = Jl 


and  the  pxp  matrix 


Lg  = X B A 
pxp 


and  by  Equation  (2) 


j,  = tr  AXB  = tr  BAX  = tr  XBA 


The  differential  of  the  trace  is 


d;t  = d tr  (AXB)  « d tr  (BAX)  = d tr  (XBA) 


di  = tr  A dX  B = tr  BAdX 
£xp  pxn  mx£  mxm 

•-  tr  (dXBA) 
pxp 


and  by  Equation  (lO)  and  Equation  (13) 

— ijT  = B A (14) 

dX 


■^tr  (AXB)  = -~tr  (BAX)  = (XBA)=  B A (15) 

3X  mxm  9X  pxp  £xp 


For  the  special  case  when  the  hi  are  scalers,  i.e.:  A and  B are  vectors 


(10) 


3?^  [pL  = b(^p)a 

By  Equation  (3)  for  in=Jl  and  B=Ira 


iiixp 

By  Equation  (3)  for  t=p  and  A=Ip 

X B 1=  B 
3X  \ pxm  m>cp  / mxp 
mxp 

In  the  above  equation  if  in=l 
^ [x(^<p)bj  = <(p)b 

Ixp 

Since  Lhe  trace  of  the  dyadic  product  is  the  inner  product 
tr  (5^<^)  = ^b^  = £ 

one  has 


4 = <*> 


where  the  matrix 


(19) 


(20) 


(21) 


(22) 


(23) 


(24) 


when  the  coordinates  are  all  independent. 


One  can  verify  all  of  the  previous  results  via  the  tedious  process 
of  partitioning.  For  example  consider  Equation  (20) 

L = X B (25) 

pxp  pxm  mxp 

and  partition  X into  its  column  vectors  and  B into  its  row  vectors  to 
obtain 


^^)b 


“1  in  ^ . 

‘ I x(p\^p)b 

i=l  ^ 


LXp)"J 


the  trace  of  Equation  (26)  is 

tr  XB  = tr  ^ ^ ( ^2^^  ) 


+ tr 


(-») 


or 


1 = tr  XB  = + 2<^x>2  t ...  + 


= h*  ...  + 


The  differential  of  I is 


di  = dil,  + d£„  + ...  + d£ 
12  m 


(26) 


(27) 

(28) 

(29) 

(30) 


where  each  of  the  jl^  is  a function  of  the  vector  5^^  or  by  Equation  (l3) 
0 

'^Vi  oi) 


Repackaging  Equation  (31) 

/3£. 


d£ 


/3£.  ^ yii 

= <-5^dx(^l  + ...  +<^^dx(p>^ 

1 n 


(32) 
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By  Equation  (31)  in  Equation  (28) 


« 


(3 


Consider  the  product  of  the  matrix  of  gradient  vectors  with  the 
matrix  of  dX  vectors  that  is 


mxp 


Clearly 


(31 


(3 


(31 


(3' 


The  other  results  can  be  sho\m  by  similar  partitioning  and  can  be 
found  in  detail  in  reference  [4]. 


B 


QUADRATIC  CASES.  Consida?  the  ixt  matrix  product 


= A X C X B 
Ixj,  ™’‘P  P’^^ 


(38) 


c*r 


= Tax  1 c x^  b 
ixa  UML  . 


(39) 


Form  the  new  matrix  of  size  mxm  from  Equation  (39)  cyclically  permuted 
via  parentheses 


^2 

mxm 


= [c  x"^  bJ 


C X Bj  AX 
mxf  ixm 


(40) 


Form  a third  matrix  of  size  pxp  by  cyclically  permuting  the  A matrix  of 
Equation  (39) 


Qg  = X C X 

pxp 


B A 

pxm  mxm  mxp  px)i  Jixp 


(41) 


The  differential  of  Equation  (40)  is 
dQ, 


‘2 
mxm 


C dx'^BAX  + Cx'^EAdX 


(42) 


The  first  matrix  on  the  right  of  Equation  (42)  under  the  trace 
transposition  and  permutation  rules  becomes , 

nt  fp 

tr  (CdX  BAX)  = tr  C X A B dX 
Using  Equation  (43)  in  the  trace  of  Equation  (42)  yields 

tr  dQj  = tr  (c'^x'^a'^b'^+Cx'^BA  ) dX 
mxm 

The  gradient  factors  of  dQ^  can  be  taken  to  be 


(43) 


(44) 
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:aiS.T~^rT'~rWtSCI- 


atrdQ 

dQ,  = ^ dX 

3X^  pxm 

VAWn  ^ 


and  by  Equation  (45)  and  Equation  (44) 

(XCx'^BA)  = c'^x’^a’^’b’’  + CX^BA  (46) 

3xT 

By  Equation  (38),  Equation  (40)  and  Equation  (43)  the  traces  are  all 
equal,  that  is 

tr  = tr  = tr  (47) 


and  the 


d tr  Q.  = tr  dQ. 
1 a. 


hence 


Hr  ~ 
"f 

3X  IJ 


i = 1,  2,  3 


^Fa  X C x'^b1=  tCx'^BA 


Jtxp  pxm  mxin  mxp  px£, 


~ QcX*  BAx[]  = c'^X^A^b'^+  Cx'^BA 


If  C is  square  and  equal  to  I,  we  obtain  by  Equation  (46),  Equation 
(49)  and  Equation  (50) 


baJ  = ^Q\xxM 
= -^Qi^BAX^  = x’’(a'^B^+BA) 


r — : ■ ' - ' ' ' ■ ■ - - 

1 

If 

J: 

1 : 

' 1 

BA  = I 

(52) 

.1  ; 

* 

then  Equation  (51)  becomes 

.1 

Cx^x3  = ■^Cxx'^]  = 2 x’’ 

3X  3X  mxp 

(53) 

;n 

For  the  special  case  when  Equation  (49)  is  a scalar,  one  obtains 

~ |j^aXCx'^b)>]  = c'^x'^^b  + CX'^^<J 
dX 

(54) 

J 

■ ) 

and  for  C=I 

( 

•.  * 

(55) 

1 

For  p=l  by  Equation  (51) 

; 1 

(56) 

For  C=I  in  Equation  (54) 

“I 

i 

(57) 

As  a final  quadratic  case  consider  the  (full  rank)  grammian-matrix 

T 

X X = G 

(58) 

and  its  inverse 

? i 

(x'^'x)"^  = g"^ 

(59) 

where 

; 

1 

\ 

] 

g“^g  = I 

(60) 

\ 

i 

1 

\ 

1 

i 

! ^ 
si 

i 

1 

1 
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) 

] 

Lb?..--. 

The  differential  of  Equation  (60)  is 


dG‘^G 


+ G"^dG 


0 


or 


dG"^  = -G'^dGG"^ 


-1  T T -1 
= -G  [dX  X+X  dX3  G 


(61) 


(62) 

(63) 


-1  -1  T -1  -IT  -1 

dG  =-G  dX  XG  - G X dXG  (64) 

The  generalized  inverse,  for  the  full-rank  case,  is  defined  as 

X*  = (X^X)"^  X^  = g"V  (65) 

mxp 

and  its  transpose  is 

X*’’  = X G"^  (66) 

pxn  pxm  mxm 

Using  Equation  (65)  and  Equation  (66)  in  Equation  (64) 

dG"^  = - g"^  dxV*^  - X*dXG'^  (67) 

The  trace  of  Equation  (67)  is 

tr  dG"^  = - tr  (dx’^X^’i'^G'^)  - tr  (6"^X*dX) 

Using  the  trace  transpose  property  on  the  first  right  hand  side  term 
of  Equation  (68) 

tr  dG"^  = - tr  (G~^X*dXtG"^X*dX)  (69) 

= - 2 tr(G~V>dX) 
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J 


or 

(x’^X)"^  = - 2 (x'^X)‘V  (71) 

3X  raxm 
mxp 


Using  Equation  (65)  in  Equation  (71) 

(x'^X)"^  = - 2 (x'^X)"^  x"^  (72) 

3X  mxm  mxp 

C.  ’’CUBIC"  CASE.  The  following  cases  do  not  involve  cubics  but  the 
matrix  X and  appear  three  times  in  the  generalized  inverse  relation 
(f\jll  rank) 

X*  = (x'^x)"^  X^  (73) 

mxp  mxm  mxp 

Consider  the  linear  form  in  X*  that  is 

Q = X*  B (74) 

mxm  mxp  pxm 

and 

dQ  = dX*B  (75) 

dQ  = d [(x'^X)'V:  B 

= d(x'^X)~^  x'^B  + (x'^X)"^  dx’^’s  (76) 

by  Equation  (^3)  Equation  (76) 

dQ  = ~ G"^  [dx'^X+x'^dX]  G"Vb  t G’^dx'^B  (77) 

or  by  Equation  (73)  in  Equation  (77) 
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t 


dQ  = - G“^[dX^X+x'^dX3X*B  ^G"^dX^B 
Using  the  relation  for  the  projector 

T 2 

X X*  = P =?=?■" 
pxm  mxp  pxp 

in  Equation  (78) 

dQ  = - G"^  dx"^  PB  - X*dXX*B  + G"^  dx'^^B 
or 

dQ  = G~^  dX*^  (I-P)  B - X*  dXX*B 

The  orthogonal  compliment  projector  is 

- -T  -2 

P = 1-  P=P  =P 

Using  Equation  (82)  and  Equation  (81) 

dQ  = G"^  dx'^PB  - XftdXX*B 

The  trace  relations  yield 

tr  dQ  = + tr(dx'^PBG"“)  - tr  (X*BX»MX) 
and  transpose-wise 

tr  dQ  = tr  ( G'^B'^fdX)  - tr  X*BX>’'dX 


or 

tr  dQ  = tr  [(  G"Vp-X*BX*)dX] 


one  obtains 

~ (X*B)  = G"^  B^P  - X*  BX* 

9X  ram 

(88) 

D.  POWERS  OF  (X*B)”.  The  following  cases  ar^e  of  higher  degree  than 
third.  Consider  n=2  then  the  Q of  Equation  (74)  has  square 

= X*BX*B 

(89) 

and 

dQ^  = dX*BX*3  + X*BdX*B 

(90) 

By  Equation  (75) 

dQ  = dX*B 

(91) 

hem-e  Equation  (91)  in  Equation  (90) 

dQ‘‘  = dQ:<*B  + X*BdQ 

(92) 

By  Equation  (89)  let 

= tr 

(93) 

then 

2 ^*^2 

dQ  = — =r  dX  = dQQ  + QdQ 

3X^ 

(94) 

The  trace  of  Equation  (92)  is 

tr  dQ^  = tr  2X*BdQ 

(95) 

By  Equation  (94)  and  Equation  (95) 

Using  dQ  given  by  Equation  (83)  in  Equation  (96) 


tr  X*BdQ  = tr  {X*B  (g"  dX  PB-X*dXX*B)} 


tr  X*BG~  dX  fB  - tr  X*BX*dXX*B 


tr  dX  (?BX5^B6~^)  - tr  X*BX*BX*dX 


That  is 


with 


r 


and  the  trace  of  Equation  (105)  is 

3 2 

tr  dQ  = 3 tr  Q dQ 

By  Equation  (83)  in  Equation  (107) 

tr  dQ®  = 3 tr  EG“^dx'^PB-X*dXQ] 


(107) 


(108) 


Using  the  commuting  and  permuting  properties  of  trace  one  obtains 


^ = 3[g-^ 


(109) 


For  n=l,2,  and  3 by  Equation  (88),  Equation  (lOl)  and  Equation  (109), 


’1  -1  T- 

= G B P - QX* 


(110) 


= 3iG“^(Q'^)^  b'^’p  - Q®X-^ 

3X  i-  J 

This  gives  the  inductive  step,  therefore  for 

Q = X*  B 
mxra  mxp  pxm 


= (x’^’x)'^ 

njxjn  mxm 


X*  = G X 


(111) 


(112) 


(113) 


(114) 


(115) 
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P = XX* 

pxp 

P = I - P 

• tr  q";  n = 

we  have 

= n C6~^(q'^)""Vp-qV0 
3X 


(116) 

(117) 

(118) 

(119) 


A number  of  special  cases  follow  from  Equation  (119).  Suppose  X is 
square  and  full  rank,  then  " 


(120) 


(121) 

(12P) 


E.  POWERS  OF  (XB)  . The  powers  of 


q"  = (XB)” 
pxp 


can  be  obtained  in  a similar  manner. 


B 

mxp 


By  Equation  (20) 


Let 


2 2 
Q = (XB)^ 

^2  = tr 


and 


dQ^  = dQQ  + QdQ 

tr  dQ^  = 2 tr  (QdQ) 

= 2 tr  QdXB  = 2 tr  (BQdX) 


3^2 

T 


= 2 BQ 


Repeating  the  arguments  as  before  one  obtains 

-i*  Ctr  (XB)”]  = n B (X  B )”"^ 

3X  mxp  pxp 


3.  "Cubic”  Forms  and  Others:  The  generalized  inverse 

X*  = x’’^  = G"^  X*^ 

(X*  B ) = (x'^x)"^  b'^'p  - X*BX* 

3X  mxp  pxm 

where  the  projectors  are 
P = I - P 
P = XX* 

G = x'^X 

^ (X*B)"  = n b'^’p  - qVo 

8X 

^ (X"^B)”  = - n (X"*B)"  X"^ 

3X^ 

= - n 


-r  (XB)”  = nB(XB)""^ 


(X")  = nx”"^ 

ax 

III*  SOME  FUNCTIONS  OF  GS.’fERALIZra  MATRIX  PRODUCTS  I>:VOLVTNG  ( X , , X* . ) 
OF  FIKIfti  0.^^■JEK.  In  this  section  we  will  be  concernea  V7ixh  matrix  functions 
of  tlie  form 

Y = B 'Z  -B  -Z  •••B  -Z  ‘B 

112  2 n n n+1 

n>o,  n an  integer  and  the  3i  scalar  matrices.  Functions  of  Y,  the  argu- 
ment, v/ill  be  examined,  such  as  f(Y)=Y,  and  certain  transcendentals 
identified  below.  The  cilgorithms  to  be  developed  are  divided  into  three 


Chen  in  his  applications  to  Walsh  functions  defines  the  Kronecker 
product  as  in  Equation  (it).  Conlisk  points  out  that  Neudecker's  papers 
are  the  only  published  papers  he  had  found  which  formalized  the  operations 
of  a and  the  "stacking  operator"  L(A),  or  as  used  by  many  vec  A where  if 
A is  mxn  and  partitioned  into  its  column  space  [note  L(a)  notation  will 
not  be  used] 


A = a(i^  , . . .a(i^ 
mxn  L 1 n-* 

then  the  column  of  column  vectors  is 

vec  A = PaCm'y  ' = v(^> 

i a(m> 
i '2 


i a(m> 


Hartwig  in  his  paper  calls  vec  X col  X,  that  is  for  (note  xi  are  column 
vectors ) 

X = [x  ,x  ,* • -x^] 

1 / n 


mm  T T 

col  X = [x^  ,x;: , • • 'X  ] = vec  X 

12  n 

The  Kronecker  product  and  the  vec  X definitions  of  Neudecker  will  be 
used  in  this  report  since  so  much  published  literature  is  available  which 
has  referenced  his  work. 

The  matrix  Ajjixn  can  be  partitioned  into  a row  of  column  vectors  or  a 
column  of  row  vectors 


f 


(7) 


(8) 


hence 


vec 


(9) 


It  is  clear  from  the  for-going,  that  how  the  coliamn  vectors  of  Equa- 
tion (6)  are  packaged  in  terms  of  the  partitioning  of  A is  quite  arbitrary, 
for  example  one  could  build  up  an  algebra  by  working  with  large  row  vectors 
made  up  as  a row  of  the  rows  of  Equation  (7). 
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Ben  Israel  and  Greville  in  their  book  define  the  matrix  Kronecker 
product  as  in  Equation  (2)  but  define  the  corresponding  vectors  by  Equa- 
tion (7)  as  a row  of  the  row  vectors 


r(X)^  = ^^)a,  ^n)a,*“^)oJ 


v(X)  5 1 


which  we  see  by  Equation  (9)  is  vec  as  used  by  Neudecker. 

In  Ben  Israel's  notation  via  page  (42)  of  reference  ( );  for  any 
X=[xij]eC’’''^^,  let  the  vector  v(X)-[v]^]eC™n  the  transpose  of  the  row 
vector  obtained  by  placing  the  rows  of  X end  to  end  with  the  first  row  on 
the  left  and  the  last  row  on  the  right.  In  other  words 

"^nCi-D+j  “ ^ij  (i=l>2,...m;j=l,2,...n) 

Using  this  system  (or  partitioning  and  packaging  procedure)  Ben  Israel 
and  Greville  then  invite  the  energetic  reader  to  verify  their  relation 
(page  42  of  their  book) 

v(AXB)  = (AhB^)  v(X)  (11) 

Clearly  the  results  of  Equation  (11)  will  be  different  from  the  Equation 
42  results  of  Neudecker,  thus  the  user  should  be  wary  of  blindly  using 
formulas  from  various  books  and  papers. 

The  relationships  between  the  indices  for  vec  X are 


A . . - V , . , V . 

ID  m(:]-l)+a 
(i=l,2, . . .m;  j=l,2,...n) 

The  Kronecker  product  of  Equation  (3)  can  be  written  in  terms  of 
indices  for  the  square  matrix  case  of  A(m,m)  and  B(n,n)  we  have 


A H B = C 
mxm  nxn  (mn)x(mn) 


[a  .]  H [b  J = [C  J 
ri  S3  kA 


where 


r , i  = 1,2, .. .m 

s,  j = 1,2,. . .n 
or  element-wise 


where 


C,  = a . b . 
kZ  ri  sj 


k = (r-1)  n + s 


( 


(: 


■ s mod  n 


and 


I = (i-1)  n + j 
5 J mod  n 


The  dyadic  product  of  a row  and  a column  vector  is  a simple  example  of  a 
Kronecker  matrix  product,  e.g. 


« B = a(ra^^)b  = a^<^)'b 


(mxl) (ixn) 


a b (19) 
n 


<^)b 


and  we  see  that  the  size  is  mxn. 


Consider  the  usual  inner  product  of  a column  and  row  vector 

<^)b  a(n>  = b a + h a + ...  + b a 
N ^ 112*-  nn 

and  the  corresponding  Kronecker  product 

4 m a>  = Cb^a>,  b2a>....b^a>] 

(ixn)  (nxl) 

= [^b^,  a)>b2  ...  a>b^: 

^)b  a a(^  = a(n'^^n)b  (20) 

or  for  m^n,  the  inner  product  is  not  defined  but  foi’  Equation  (20)  we 
have 

<^)b  a a(m>  = [b^a(m'>,  ...  b^a(^J 
(Ixn)  (mxl)  mxn, 

.^n)b  a a(i^  = a(:^<^n)b  = a(^a^)b  (21j 

which  is  the  same  as  Equation  (19). 


The  following  pages  will  derive  a number  of  Kronecker  product  rela- 
tionships and  then  summarize  the  bulk  of  those  to  be  used  in  the  report 
Consider  an  (m^n)  matrix  X 


X = I X = El  x(i^,  I 3^  I 
m^n  mxm  mxn  1 2 n 


or 


vec  X 


-:>• 


I x> 
n, 


I 

m’^m 

0 I 

mxm 


I 

mxm 


r a 

x(9> 

x(i^ 


11 


or 


vec  X = [ I B I J vec  X 
\ nxn  mxm  / 


since 


I a I 
nxm  mxm 


1 I 
mxm 


1 I 
mxm 


1 I 
mxm 


(22 


(23 


(25: 


Equation  (22)  can  also  be  written  as 
X = X I 


mxn  mxn  nxn 


(26 


Consider  the  first  column  vector  of  Equation  (42) 


and  using  Equation  (2) 


I 


vec  (AXB)  = (B  aA)  vec  X 

By  Equation  (48)  when  B=I  we  obtain  Equation  (33). 
By  Equation  (48)  for  A=I  we  obtain 

_ /-..T  f \ 


vec  (XB)  = (B  al)  vec  X 
Consider  next  the  relations 


A C a B.D  =(AaB)(CaD) 
(kxm)(mxr)  (sxin)(nxt)  (kxm)  s^n  mxr  n^t 


AC  aBD  = (AbB)  (CaD) 
kxr  sxt  (ksxinn)(mnxrt) 


ACaBD  = ( AbB) (CaD) 
ksxrt  ksxrt 


where  the  matrix  sizes  are  indicated  at  each  step. 


The  proof  of  Equation  (50)  will  be  denonstrated  for  2x2  matrices  by 
construction 


(A«B)(CaD)  = ra,,B  a,^Birc,,D  C, 


11"  “12  11  12 

a„,B  a„.,B  C„,D  C„„D 


‘22"J  L''2r 


^®11^12‘^®12^22^®°1 


^®21^1l'’’^22^21^®°  ^®2l‘^12''’^22^22^®°J 


■ r®ll^ll'^*12^21  ®11^12''’®12^22 


a„,C, - +a--C.,  a.  C,.+a„-C., 
- 21  11  22  21  21  12  22  22 


a mm  ■ n— 


or 

(A«B)(C«D)  = ACaBD  (54) 

The  relations  of  Equation  (50)  will  be  used  to  prove  (Lancaster 
page  258) 

(AaB)"^  = a'^hB"^  (55) 

Note  the  inverse  matrices  are  not  commuted  as  in  the  standard  product 
inverse  where 

(AB)"^  = b"^A~^  (56) 

or  the  full-rank-factor  case  for  the  generalized  inverse 
, * jV  is 

(A  B)  = B A (57) 

mxil  Jlxn  ilxm 

where  * means  generalized  inverse  and  the  rank  of  A and  B is  £. 

By  Equation  (55) 

(AbB)(AhB)'^  = (AiaB)(A~^aB"^)  (58) 

By  Equation  (54)  for  square  invertible  matrices 

(AaB)(A“^HB"^)  = AA"^bBB~^  = I al  = I (59) 

™ (mn)(mn) 

where  A is  m^m  and  B is  n^n  square  matrices. 

Using  Equation  (59)  in  Equation  (58) 

(AaB)(AaB)"^  = I al  = I (60) 

m n mn 

which  establishes  Equation  (55). 

The  inverse  of  the  Kronecker  product  of  two  matrices  of  Equation  (55) 
can  be  derived  in  other  ways,  some  of  which  are  of  interest,  e.g.  consider 
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A«B=ra^^B  ... 


AhB  = 


= (AaDdaB) 
and  by  Equation  (56) 

(AaB)"^  = (laB)'^  (AaD'l 
It  is  obvious  that 


= rB 


-1  ^-1 


0 


It  is  not  obvious  what  the  inverse  of  Aal  is  where  in  open  form 

(Aal)  = ra.^I  ...  a I *|  (65) 

11  m 1mm 


Define 


-1  * r * 

A =A  =ra^^ 


Tw 


Applying  Equation  (55)  to  Equation  (65) 


(AhI)  ^ = a = 


* 


* 

a I 

mi 


* 

a,  I 

Im 


ft 

a I 
ra 

m 


Using  Equation  (68)  and  Equation  (65) 
1 


(AiaI)(A!aI) 


"11^ 


a I 
L "'l 


a,  I' 

Im 


a I 
m 


•1 

ra,  I 

• 

a,  Ii 

/l 

Im 

ft 

ft 

a I 

a I 

m. 

m 

L-  1 

m -J 

ft  ft 


ft 


m 1 m^m 


ft  ft 

m,  1 
1 m 


Note  that  by  Equation  ( gg)  the  biorthogonal  relations  are 


which  should  be  contrasted  with  the  standard  matrix  product 


(AB)"^  = 


where  conmuting  takes  place  as  in  Equation  (56). 
By  Equation  (61)  and  Equation  (62) 


AaB  = (AaI)(lBB) 


■f^ll^  ^12^ 


Using  Equation  (77)  concept  in  Equation  (78) 


(AaB)"^  = r 


^1^  ^21^ 


0 B 


^ml'l 


0 


'[*1/  ^2/  •••  \/l 


a B 
ram 


f..  «•  y* 


w 


(AhB)^  = (A^bI  )(I  hb"^)  (82) 

m n 


and  by  Equation  (54) 

(AhB)^  = A^hb'^  (83) 

Now  that  the  flavor  of  the  partitioning  is  established  it  is  easy  to 
establish  the  following  additional  properties. 

(A+B)  b (C+D)  = (a  a C)  + (AbD)  + (BaC)  + (BaD)  (84) 

and  also 

A a (BaC)  = (AaB)  a C (85) 

also  by  Equation  (87) 

(A+B)  a C = (AaC)  + (BaC)  (86) 

and 

B a (C+D)  = (BaC)  + (BaD)  (87) 

By  Equation  (78) 


A a B = (Aal  )(I  ®B)  (88) 

(mxm)  nxn  ” 

We  will  show  that  (Aal^)  and  (1^86)  commute. 


A a B 
mxm  nxn 


11  n 


a,  I 

Im  n 


I 

nx 


a 


ml  n 


a I 
nm  n 


I B 
n nxn 


B 

Im  nxn 


or 


L ml  nxn 


a B 
mm  nxn 


B 


ll^n 


L 


^ml^n 


A a B = (I  HB)(AaI  ) 
m n 


and  by  Equation  (88)  and  Equation  (92) 

A a B = (Aal  )(I  bB)  = (l  bB)(AbI  ) 


r 


Appendix  (H) 


Dirac  Delta  Function 


The  Dirac  Delta  Function  is  used  to  obtain  the  variance  Riccatti  E- 
quations  for  the  continuous  case;  hence  some  considerations  are  given  to 
this  function  in  this  appendix. 


The  mean  value  theorem  and  the  fundamental  theorem  of  the  intergral 
Calculus  are  stated  without  proof  from  ref  (78). 


Mean  Value  Theorem.  If  the  function  f(t) 

( i)  is  continuous  in  the  closed  interval  a^  t fb 

(ii)  has  a derivative  at  every  interior  point; 
then  at  some  interior  point  C of  the  interval 


f(b)  - f(a) 


b - a 


= f'(0 


(1) 


Fig  (1)  Graph  of  mean- value  Theorem. 


The  simple  geometric  interpretation  of  the  mean  value  theorem  is  rep- 
resented by  the  curve  of  Fig  (1).  The  straight  line  chord  from  the  point 
to  the  tangent  to  the  curve  at  same  point  (C»  f(C)). 


If 


b=a+h 

then  one  can  write 

C=a+0h 

where 

O<0<1 

the  mean  value  theorem  now  takes  the  form 


(2) 


(3) 


f(a+h)  - f(a)  = h f'(a+0h) 

O<0<1 


(4) 


Fundamental  Theorem  of  The  Integral  Calculus. 

If  f(t)  is  integrable  in  (a,b)  and  F(t)  is  any  function  having  f(t)  as 
derivative,  then. 

a 


/“f(t)dt  = F(b)  - F(a)  = F(t)l^ 


(5) 
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Dirac  Function 


Consider  the  Dirac  Function  (t)  defined  to  be  zero  if  t=0,  in  such 
a way  that  „ 

f (t)dt  = 1 (6) 

wOO 

Equation  (6)  is  a concise  manner  of  expressing  a function  that  is 
very  large  in  a very  small  region  and  zero  outside  this  region  and  has 
a unit  integral. 


Consider  a function  6(t-a)  that  has  the  following  values 

^0  t^a 

6(t-a)  =>_!  a<t<a+h 

( 

0 a+h<t 


(7) 


where  h may  be  made  as  small  as  we  please.  The  function  6(t-a)  has  the 
property  of  Eq  (6)  for 


a+h 

6(t-a)dt 

= /6(t-a)dt 

(8) 

a 

a+h 

a+h 

— tj  =—  = 1 

h ' h 

(9) 

a 

a 

Consider  next  the  integral  of  the  function 

/ ”f(t)  6 (t-a)  dt 
— 00 

a+h 

= f f(t)  6 (t-a)  dt 
a 

, a+h  a+h 

=i/  f(t)dt  =iF(t)| 


where 


F'(t)  = d F(t)  = f(t) 
dt 


(10) 


(11) 


(12) 


Evaluating  Eq  (11)  at  the  limits 


a+h 

/ f(t)  6(t-a)dt  = ^F(a+h) 
a 


F(a)] 


(13) 
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By  the  mean-value  theorem  of  Eq  (1) 


F(a+h)  - F(a)  = h F"(a+0h) 
O<0<1 

By  Eq  (12)  in  Eq  (14) 

a+h  ^ 

/ f(t)  6(t-a)dt  = r-  f(a+0h) 
h 
a 

in  the  limit  for  f(t)  continuous 

lim  f(a+0h)  = f(a) 
h-K) 

hence  Eq  (16)  in  Eq  (10) 

/ °°f(t)  6(t-a)dt  = f(a). 
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